-- Logs begin at Tue 2022-08-30 13:59:44 UTC, end at Tue 2022-08-30 15:06:49 UTC. -- Aug 30 14:29:08.331118 dv-cinder-828805-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 30 14:29:08.339290 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 30 14:29:08.343088 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 30 14:29:08 2022] *** Aug 30 14:29:08.343088 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 30 14:29:08.343088 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 30 14:29:08.343088 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: nodename: dv-cinder-828805-5-iscsi Aug 30 14:29:08.343088 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: machine: x86_64 Aug 30 14:29:08.343088 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: clock source: unix Aug 30 14:29:08.343088 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: pcre jit disabled Aug 30 14:29:08.343088 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: detected number of CPU cores: 4 Aug 30 14:29:08.343088 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: current working directory: / Aug 30 14:29:08.343088 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: detected binary path: /usr/bin/uwsgi-core Aug 30 14:29:08.343088 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: your processes number limit is 47680 Aug 30 14:29:08.343088 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: your memory page size is 4096 bytes Aug 30 14:29:08.343088 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: detected max file descriptor number: 2048 Aug 30 14:29:08.343088 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: lock engine: pthread robust mutexes Aug 30 14:29:08.343088 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: thunder lock: enabled Aug 30 14:29:08.343088 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 30 14:29:08.343088 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 30 14:29:08.381882 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: Python main interpreter initialized at 0x55a805c3f9c0 Aug 30 14:29:08.381882 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: python threads support enabled Aug 30 14:29:08.381882 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: your server socket listen backlog is limited to 100 connections Aug 30 14:29:08.381882 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: your mercy for graceful operations on workers is 90 seconds Aug 30 14:29:08.382315 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: mapped 403077 bytes (393 KB) for 2 cores Aug 30 14:29:08.382315 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: *** Operational MODE: preforking *** Aug 30 14:29:08.382598 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: *** uWSGI is running in multiple interpreter mode *** Aug 30 14:29:08.382598 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: spawned uWSGI master process (pid: 81722) Aug 30 14:29:08.382831 dv-cinder-828805-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 30 14:29:08.383920 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: spawned uWSGI worker 1 (pid: 81732, cores: 1) Aug 30 14:29:08.383920 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: spawned uWSGI worker 2 (pid: 81733, cores: 1) Aug 30 14:29:08.383920 dv-cinder-828805-5-iscsi devstack@keystone.service[81722]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 30 14:29:12.712283 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 14:29:12.712283 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: from cryptography.utils import int_from_bytes Aug 30 14:29:12.713105 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 14:29:12.713105 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: from cryptography.utils import int_from_bytes Aug 30 14:29:12.723888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 14:29:12.723888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: from cryptography.utils import int_from_bytes Aug 30 14:29:12.724689 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 14:29:12.724689 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: from cryptography.utils import int_from_bytes Aug 30 14:29:12.763909 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 14:29:12.764129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 14:29:12.764290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] command line args: [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 14:29:12.764553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 14:29:12.764726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ================================================================================ {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 14:29:12.764981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] admin_token = **** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.765224 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] config_dir = [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.765403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.765580 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] config_source = [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.765756 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] debug = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.765952 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.766712 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.766712 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.766712 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.766959 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.767161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] list_limit = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.767332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] log_config_append = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.767685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.767685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] log_dir = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.767920 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] log_file = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.768240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.768414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.768595 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.768851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.769091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.769266 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.769508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.769748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.769981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.770202 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.770429 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.770661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.770900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.771138 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.771413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.771920 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.772116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] publish_errors = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.772348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.772578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.772751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.772997 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.773230 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.773462 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] standard_threads = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.773692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.773930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.774078 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.774254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] use_journal = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.774428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] use_json = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.774595 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] use_stderr = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.774762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] use_syslog = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.774984 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.775068 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 14:29:12.775183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 14:29:12.775273 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.775378 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] command line args: [] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 14:29:12.775468 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.775654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 14:29:12.775735 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.775831 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ================================================================================ {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 14:29:12.775919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.776121 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] admin_token = **** {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.776229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.776365 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] config_dir = [] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.776478 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.776558 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.776647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.776724 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] config_source = [] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.776847 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.776940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] debug = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.777071 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.777190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.777307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.777395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.777488 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.777569 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.777738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.777828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.777986 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.778102 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.778239 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.778345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] list_limit = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.778441 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] log_config_append = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.778518 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.778589 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.778669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.778745 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] log_dir = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.778862 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] log_file = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.778944 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.779100 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.779203 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.779353 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.779448 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.779611 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.779696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.779807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.779946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.780125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.780268 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.780351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.780437 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.780537 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.780628 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.780774 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.780858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.780940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.781016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.781136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.781261 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.781386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.781532 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.781669 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.781787 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.781888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.781990 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.782072 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.782150 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.782222 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.782291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.782506 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.782737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.782831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.782903 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] publish_errors = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.782971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.783142 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.783288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.783465 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.783674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.783754 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.783819 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.783980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.784187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.784269 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.784564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.784699 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.784823 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.784963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] standard_threads = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.785087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.785321 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.785441 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.785561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.785642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.785713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.785713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] use_journal = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.785848 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.785919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] use_json = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.785994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.786063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] use_stderr = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.786131 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.786205 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] use_syslog = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.786293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.786416 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:29:12.786545 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.786660 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.786775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.786897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.787036 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.787139 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.787218 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.787288 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.787362 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.787443 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.787514 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.787639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.787721 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.787888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.787968 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.788155 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.788238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.788382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.788496 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.788623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.788747 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.788835 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.788915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.789060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.789185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.789302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.789433 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.789555 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.789791 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.789876 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.789954 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.790044 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.790160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.790295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.790413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.790540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.790671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.790757 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.790898 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.791130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.791225 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.791342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.791454 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.791576 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.791708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.791837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.791948 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.792085 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.792159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.792272 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.792402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.792526 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.792624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.792725 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.792820 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.792925 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.793056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.793191 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.793404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.793504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.793608 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.793707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.793836 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.793931 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.794006 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.794122 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.794220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.794388 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.794473 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.794554 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.794626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.794739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.794879 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.794962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.795116 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.795206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.795366 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.795496 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.795636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.795748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.795825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.795941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.796064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.796141 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.796255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.796336 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.796486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.796618 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.796709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.796812 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.796942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.797329 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.797406 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.797523 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.797607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.797680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.797793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.797866 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.797979 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.798053 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.798174 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.798250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.798327 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.798398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.798511 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.798585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.798664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.798737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.798917 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.798999 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.799181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.799265 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.799410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] auth.external = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.799640 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.800054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.800153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.800299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.800381 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.800529 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] auth.password = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.800612 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.800755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] auth.token = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.800837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.800987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.801071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.801217 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.801306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.801459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.801542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.801685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.801766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.801916 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.802000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.802149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.802223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.802371 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.802453 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.802603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] credential.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.802700 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.802845 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.802928 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.803081 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.803160 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.803325 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.803417 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.803565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.803649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.803803 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.803887 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.804057 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.804155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.804296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.804377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.804531 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.804614 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.804762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.804854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.805007 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.805096 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.805244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.805328 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.805480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.805571 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.805720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.805802 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.805956 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.806047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.806202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.806296 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.806431 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] federation.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.806522 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.806663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.806753 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.806894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.806975 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.807127 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.807218 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.807349 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.807430 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.807591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.807674 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.807820 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.807911 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.808089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.808175 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.808323 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.808405 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.808558 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.808655 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.808795 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.808876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.809029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.809109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.809257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.809346 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.809495 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.809585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.809726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.809816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.809955 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.810053 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.810184 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.810268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.810418 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.810499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.810642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.810732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.810874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.810957 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.811106 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.811187 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.811351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.811440 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.811578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.811661 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.811803 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.811886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.812062 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.812161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.812288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.812371 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.812527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.812610 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] auth.external = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.812768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.812860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.813021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.813105 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.813261 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.813345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.813500 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] auth.password = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.813589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.813733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] auth.token = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.813816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.813964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.814045 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.814190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.814285 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.814421 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.814505 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.814669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.814779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.814936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.815280 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.815516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.815596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.815760 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.815841 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.815981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.816118 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.816241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.816324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] credential.caching = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.816482 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.816565 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.816711 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.816796 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.816953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.817035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.817195 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.817303 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.817422 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.817556 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.817638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.817796 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.817879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.818049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.818140 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.818304 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.818391 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.818543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.818670 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.818772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.818863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.819003 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.819087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.819246 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.819328 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.819488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.819579 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.819715 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.819798 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.819942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.820080 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.820208 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.820304 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] federation.caching = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.820450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.820533 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.820682 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.820763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.820912 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.821002 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.821145 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.821227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.821370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.821452 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.821604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.821694 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.821855 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.821937 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.822081 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.822164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.822323 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.822414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.822563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.822645 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.822801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.822883 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.823031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.823121 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] identity.caching = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.823262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.823345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.823491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.823573 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.823719 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.823810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.823951 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.824055 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.824201 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.824283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.824442 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.824536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.824679 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.824761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.824915 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.824996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.825157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.825255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.825385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.825466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.825609 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.825689 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.825835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.825924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.826068 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.826149 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.826299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.826381 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.826531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.826621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.826761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.826850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.826999 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.827081 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.827240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.827324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.827471 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.827553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.827718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.827800 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.827944 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.828045 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.828207 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.828288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.828436 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.828517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.828666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.828750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.828897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.828979 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.829133 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.829278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.829368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.829518 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.829602 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.829747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.829829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.829979 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.830072 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.830217 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] resource.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.830351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.830617 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.830872 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.831168 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.831260 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.831448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.831531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.831691 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.831774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.831925 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.832049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.832194 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.832276 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.832428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.832509 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.832664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.832763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.832899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] role.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.832983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.833138 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.833219 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] role.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.833365 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.833530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.833614 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.833791 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.833873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.834046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.834129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.834287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.834370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.834524 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.834605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.834762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.834843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.835002 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.835085 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.835242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.835322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.835824 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.835908 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.836090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.836176 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.836326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.836407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.836565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.836647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.836806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.836891 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.837046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.837242 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.837325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.837473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.837564 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.837716 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.837800 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.837959 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.838051 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.838200 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.838284 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.838437 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.838518 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.838662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.838743 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.838902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.838983 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.839135 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.839217 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.839367 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.839448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.839588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.839663 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.839807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.839890 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.840065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.840153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.840295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.840378 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.840545 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.840628 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.840789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.840871 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.841023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.841108 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.841255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.841337 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.841480 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.841561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.841711 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] token.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.841795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.841944 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.842026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.842186 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.842422 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.842504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.842669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.842750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.842900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.842980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.843140 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.843222 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.843375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.843456 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.843612 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.843693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.843849 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.843932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.844100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.844182 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.844334 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.844416 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.844565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.844648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.844805 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.844886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.845046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] resource.caching = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.845126 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.845286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.845377 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.845524 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.845607 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.845759 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.845839 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.846122 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.connection = **** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.846221 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.846429 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.846679 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.846818 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.846903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.847062 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.847174 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.847345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.847425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.847541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.847624 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.847782 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.847874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.848076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.848164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] role.caching = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.848273 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.848356 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] role.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.848504 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.848598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.848741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.848823 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.848972 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.849055 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.849213 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.849303 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.849442 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.849526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.849681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.849764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.849906 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.849997 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.850140 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.850223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.850368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.850450 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.850521 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 14:29:12.850624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.850872 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.851119 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.851368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.851616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.852073 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.852318 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.852570 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.852822 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.853073 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.853324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.853559 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.853815 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.854067 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.854307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.854541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.854777 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.855008 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.855261 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.855517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.855765 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.856042 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.856261 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.856494 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.856730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] token.caching = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.856969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.857219 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.857456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.857714 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.857963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.858202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.858446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.858691 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.858940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.859177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.859428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.859663 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.859921 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.860222 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.860455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.860703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.860958 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.861227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.connection = **** {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.861470 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.861941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.862825 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.863076 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.863317 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.863561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.863802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.864064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.864316 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.864555 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.864802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.865041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.865291 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.865526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.865770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.866004 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.866244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.866479 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:29:12.866654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 14:29:12.880535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:12.880717 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:12.880881 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:12.889488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:12.893704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:12.894987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:12.895229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:12.904377 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:12.918393 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:12.918630 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:12.923903 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:12.924269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:12.973490 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:12.974845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.023401 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.023721 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.026383 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.026593 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.188741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.189095 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.189095 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.189511 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.189511 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.189877 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.201175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.201428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.201612 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.201835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.202052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.202262 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.214334 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.214550 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.214768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.214984 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.215196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.215395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.215606 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.225451 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.225758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.225999 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.226274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.226492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.226723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.227075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.227539 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.236583 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.240919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.241145 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.241368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.241586 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.241799 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.242005 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.242208 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.247977 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.248226 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.248447 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.248656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.248868 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.249075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.249290 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:29:13.259741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.259981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 14:29:13.260313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.260534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 14:29:13.260769 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.261031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.261286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.261536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.263187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.263391 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 14:29:13.263740 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.263927 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 14:29:13.264218 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.264475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.264730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.264990 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.271253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.271530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.271795 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.272096 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.274915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.275209 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.275476 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.275744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.279167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.282078 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.282356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.282649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.282793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.282953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.283191 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.283499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.283807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.284731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.284731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.284731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.285008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.286002 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.286284 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.286572 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.286857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.287131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.287391 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.287660 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.287930 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.288223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.288496 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.288756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.307514 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.309027 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.309312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.311377 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.312875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.313155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.313934 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.314216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.314493 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.317858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.318144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.318422 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.321192 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.321486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.324803 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.325093 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.325559 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.325849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.326137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.326396 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.326702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.328929 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.329170 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.329317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.329483 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.329611 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.329741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.329988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.330255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.330783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.331060 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.331436 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.345824 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.346199 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.349442 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.349727 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.350109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.353257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.353552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.353678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.356862 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.357160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.357973 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.361291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.361411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.364573 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.364700 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.364974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.365251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.365532 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.365817 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.366200 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.367765 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.368101 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.368384 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.368668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.368934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.369207 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.383055 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.383961 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.383961 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.383961 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.385308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.385600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.385863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.386120 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.390618 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.392314 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.393157 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.393914 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.394180 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.394771 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.396334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.396606 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.396850 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.397131 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.397409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.397688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.397987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.399212 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.399499 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.399788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.400100 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.400485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.409605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.409893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.410167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.410446 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.410717 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.410997 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.411285 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.411916 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.412197 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.412441 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.412727 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.413109 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.413383 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.413658 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.427102 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.429092 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.430155 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.432017 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.433051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.434476 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.434875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.435943 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.436257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.436486 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.436718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.437755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.438029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.438428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.443366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.444821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.445051 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.446538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.447667 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.447951 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.448266 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.448538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.449423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.449817 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.450108 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.450401 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.456672 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.456962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.457235 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.458212 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.458503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.458781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.464509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.464801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.465096 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.465501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.465743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.465881 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.466120 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:29:13.466262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.466412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.466557 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.466694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.466793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.466937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.467086 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.467221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.467300 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.467448 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.467532 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.467706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.467794 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.467983 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.468151 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.468320 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.468600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.468869 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.469119 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.469391 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.469649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:29:13.502207 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a805c3f9c0 pid: 81732 (default app) Aug 30 14:29:13.504191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a805c3f9c0 pid: 81733 (default app) Aug 30 14:29:13.506097 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2c7e0-6350-482f-a5a5-7f5299fd36d1 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:13.506377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2c7e0-6350-482f-a5a5-7f5299fd36d1 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:13.506645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2c7e0-6350-482f-a5a5-7f5299fd36d1 None None] PATH_INFO: `/v3` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:13.511001 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Tue Aug 30 14:29:13 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:20.350950 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad663a6-87e2-48ed-8b54-13d07d335aba None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:20.351566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad663a6-87e2-48ed-8b54-13d07d335aba None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:20.351566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad663a6-87e2-48ed-8b54-13d07d335aba None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:20.355948 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:20.362659 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcd82f-deca-44cb-a8d9-70b90bd74f78 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:20.363667 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcd82f-deca-44cb-a8d9-70b90bd74f78 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:20.363667 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcd82f-deca-44cb-a8d9-70b90bd74f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:20.444999 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: INFO dbcounter [None req-fddcd82f-deca-44cb-a8d9-70b90bd74f78 None None] Registered counter for database keystone Aug 30 14:29:20.466791 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG oslo_db.sqlalchemy.engines [None req-fddcd82f-deca-44cb-a8d9-70b90bd74f78 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81732) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 14:29:20.476119 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writer thread running {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 14:29:20.696858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG passlib.handlers.bcrypt [None req-fddcd82f-deca-44cb-a8d9-70b90bd74f78 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81732) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 14:29:20.697312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG passlib.handlers.bcrypt [None req-fddcd82f-deca-44cb-a8d9-70b90bd74f78 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81732) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 14:29:20.758432 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-fddcd82f-deca-44cb-a8d9-70b90bd74f78 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:20.760118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-fddcd82f-deca-44cb-a8d9-70b90bd74f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:20.919694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 558 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 14:29:21.239694 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad663a6-87e2-48ed-8b54-13d07d335aba None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:21.313250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: INFO dbcounter [None req-1ad663a6-87e2-48ed-8b54-13d07d335aba None None] Registered counter for database keystone Aug 30 14:29:21.339916 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG oslo_db.sqlalchemy.engines [None req-1ad663a6-87e2-48ed-8b54-13d07d335aba None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81733) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 14:29:21.347334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writer thread running {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 14:29:21.482744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad663a6-87e2-48ed-8b54-13d07d335aba None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:21.488319 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13357a-a32f-45c3-9481-a653cf43e98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:21.488988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13357a-a32f-45c3-9481-a653cf43e98d admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:21.489269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13357a-a32f-45c3-9481-a653cf43e98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:21.489539 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13357a-a32f-45c3-9481-a653cf43e98d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:21.499570 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13357a-a32f-45c3-9481-a653cf43e98d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:21.553032 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5b13357a-a32f-45c3-9481-a653cf43e98d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 14:29:21.553333 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13357a-a32f-45c3-9481-a653cf43e98d admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:21.568021 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Tue Aug 30 14:29:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:29:21.574517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddcd82f-deca-44cb-a8d9-70b90bd74f78 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:21.593762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddcd82f-deca-44cb-a8d9-70b90bd74f78 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:21.599630 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c50c9-faab-4a2e-973e-d6b9322e4809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:21.600867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c50c9-faab-4a2e-973e-d6b9322e4809 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:21.601147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c50c9-faab-4a2e-973e-d6b9322e4809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:21.601414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c50c9-faab-4a2e-973e-d6b9322e4809 admin admin] PATH_INFO: `/v3/projects/f9ccb338f4e0425cad7e44cb87838ce1` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:21.603217 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c50c9-faab-4a2e-973e-d6b9322e4809 admin admin] RBAC: Authorizing `identity:get_project(project_id=f9ccb338f4e0425cad7e44cb87838ce1)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:21.669040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c93c50c9-faab-4a2e-973e-d6b9322e4809 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 14:29:21.669357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c50c9-faab-4a2e-973e-d6b9322e4809 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:21.674383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Tue Aug 30 14:29:21 2022] GET /identity/v3/projects/f9ccb338f4e0425cad7e44cb87838ce1 => generated 344 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:23.709431 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b36b2-347f-4341-bb2a-0dd01c29a9d0 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:23.710381 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b36b2-347f-4341-bb2a-0dd01c29a9d0 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:23.710381 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b36b2-347f-4341-bb2a-0dd01c29a9d0 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:23.714489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:23.721519 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa93cae-5ff2-4fab-907d-e1a56b339bb2 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:23.722296 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa93cae-5ff2-4fab-907d-e1a56b339bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:23.722296 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa93cae-5ff2-4fab-907d-e1a56b339bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:23.746321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-daa93cae-5ff2-4fab-907d-e1a56b339bb2 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:23.747921 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-daa93cae-5ff2-4fab-907d-e1a56b339bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:23.758443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 14:29:24.070183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b36b2-347f-4341-bb2a-0dd01c29a9d0 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:24.084160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b36b2-347f-4341-bb2a-0dd01c29a9d0 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:24.089546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd7453-c45b-4aa1-9962-8e5fa1f88fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:24.090493 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd7453-c45b-4aa1-9962-8e5fa1f88fcb admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:24.090813 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd7453-c45b-4aa1-9962-8e5fa1f88fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:24.091127 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd7453-c45b-4aa1-9962-8e5fa1f88fcb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:24.101072 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd7453-c45b-4aa1-9962-8e5fa1f88fcb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:24.102960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd7453-c45b-4aa1-9962-8e5fa1f88fcb admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:24.113007 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Tue Aug 30 14:29:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:29:24.119272 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa93cae-5ff2-4fab-907d-e1a56b339bb2 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:24.131338 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa93cae-5ff2-4fab-907d-e1a56b339bb2 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:24.137219 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084788b4-b7ec-4d6a-88d1-881bf68cf488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:24.137894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084788b4-b7ec-4d6a-88d1-881bf68cf488 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:24.138171 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084788b4-b7ec-4d6a-88d1-881bf68cf488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:24.138441 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084788b4-b7ec-4d6a-88d1-881bf68cf488 admin admin] PATH_INFO: `/v3/users/5f1827dde46f4915bfe3277a281b43bd` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:24.139917 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084788b4-b7ec-4d6a-88d1-881bf68cf488 admin admin] RBAC: Authorizing `identity:get_user(user_id=5f1827dde46f4915bfe3277a281b43bd)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:24.141854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084788b4-b7ec-4d6a-88d1-881bf68cf488 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:24.144027 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Tue Aug 30 14:29:24 2022] GET /identity/v3/users/5f1827dde46f4915bfe3277a281b43bd => generated 247 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:26.178588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e6d54-9623-430c-a1b2-7c7eb44c4420 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:26.179435 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e6d54-9623-430c-a1b2-7c7eb44c4420 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:26.179435 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e6d54-9623-430c-a1b2-7c7eb44c4420 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:26.180509 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:26.186253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be0439-b463-47d4-b78d-76b97bdb688d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:26.187074 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be0439-b463-47d4-b78d-76b97bdb688d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:26.187074 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be0439-b463-47d4-b78d-76b97bdb688d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:26.207363 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-20be0439-b463-47d4-b78d-76b97bdb688d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:26.208699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-20be0439-b463-47d4-b78d-76b97bdb688d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:26.213078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9dadf-75ea-4956-8eb6-9eebc4983ee0 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:26.213508 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9dadf-75ea-4956-8eb6-9eebc4983ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:26.213938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9dadf-75ea-4956-8eb6-9eebc4983ee0 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:26.215446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:26.224365 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d2fd4-a806-4b99-9e4b-80b25b85e0ae None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:26.224769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d2fd4-a806-4b99-9e4b-80b25b85e0ae None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:26.225123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d2fd4-a806-4b99-9e4b-80b25b85e0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:26.275533 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG passlib.handlers.bcrypt [None req-ca1d2fd4-a806-4b99-9e4b-80b25b85e0ae None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81733) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 14:29:26.276068 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG passlib.handlers.bcrypt [None req-ca1d2fd4-a806-4b99-9e4b-80b25b85e0ae None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81733) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 14:29:26.302332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:26.326071 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-ca1d2fd4-a806-4b99-9e4b-80b25b85e0ae None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:26.327784 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-ca1d2fd4-a806-4b99-9e4b-80b25b85e0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:26.439090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:26.577075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be0439-b463-47d4-b78d-76b97bdb688d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:26.620249 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:26.711343 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d2fd4-a806-4b99-9e4b-80b25b85e0ae None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:26.781775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be0439-b463-47d4-b78d-76b97bdb688d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:26.837265 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d2fd4-a806-4b99-9e4b-80b25b85e0ae None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:26.838434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7f5c3-dc78-480e-9143-a65dd1093c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:26.839126 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7f5c3-dc78-480e-9143-a65dd1093c75 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:26.839400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7f5c3-dc78-480e-9143-a65dd1093c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:26.839666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7f5c3-dc78-480e-9143-a65dd1093c75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:26.841352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:26.869021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7f5c3-dc78-480e-9143-a65dd1093c75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:26.881204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-19a7f5c3-dc78-480e-9143-a65dd1093c75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:26.881204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:26.881204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:26.881204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:26.881204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:26.881204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:26.881204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:26.881204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:26.881204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:26.881204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:26.881204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:26.881204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:26.881204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:26.881204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:26.881204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:26.881204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:26.883163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:26.883163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:26.883163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:26.883163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:26.883163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:26.883163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:26.883163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:26.883163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:26.883163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:26.883163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:26.883163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:26.883163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:26.883163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:26.883163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:26.883163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:26.883163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:26.885050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:26.885050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:26.885050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:26.885050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:26.885050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:26.885050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:26.885050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:26.885050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:26.885050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:26.885050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:26.885050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:26.885050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:29:26.885050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:29:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:26.891253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7f5c3-dc78-480e-9143-a65dd1093c75 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:26.896375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506700b-ee1b-47c5-80d6-2a6845a14486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:26.897434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506700b-ee1b-47c5-80d6-2a6845a14486 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:26.897749 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506700b-ee1b-47c5-80d6-2a6845a14486 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:26.898059 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506700b-ee1b-47c5-80d6-2a6845a14486 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:26.996420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506700b-ee1b-47c5-80d6-2a6845a14486 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:26.998310 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506700b-ee1b-47c5-80d6-2a6845a14486 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:27.003973 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:27.030314 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7f5c3-dc78-480e-9143-a65dd1093c75 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:27.074426 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ab61-12dd-4e6f-9b0e-697eab159a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:27.075456 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ab61-12dd-4e6f-9b0e-697eab159a3c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:27.075931 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ab61-12dd-4e6f-9b0e-697eab159a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:27.076437 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ab61-12dd-4e6f-9b0e-697eab159a3c None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:27.077467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140ab61-12dd-4e6f-9b0e-697eab159a3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:27.078009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140ab61-12dd-4e6f-9b0e-697eab159a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:27.080680 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140ab61-12dd-4e6f-9b0e-697eab159a3c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:27.102793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 8/15] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:27.108763 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ab61-12dd-4e6f-9b0e-697eab159a3c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:27.129126 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 8/16] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:27.136081 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506700b-ee1b-47c5-80d6-2a6845a14486 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:27.141154 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:27.257972 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ab61-12dd-4e6f-9b0e-697eab159a3c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:27.274575 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506700b-ee1b-47c5-80d6-2a6845a14486 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:27.312012 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14c264-0333-41f3-a21f-a252a7d5b1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:27.313039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14c264-0333-41f3-a21f-a252a7d5b1af None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:27.313485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14c264-0333-41f3-a21f-a252a7d5b1af None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:27.313895 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14c264-0333-41f3-a21f-a252a7d5b1af None admin] PATH_INFO: `/v3/users/5f1827dde46f4915bfe3277a281b43bd` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:27.315788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14c264-0333-41f3-a21f-a252a7d5b1af None admin] RBAC: Authorizing `identity:get_user(user_id=5f1827dde46f4915bfe3277a281b43bd)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:27.318371 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14c264-0333-41f3-a21f-a252a7d5b1af None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:27.324177 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 9/17] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:27 2022] GET /identity/v3/users/5f1827dde46f4915bfe3277a281b43bd => generated 247 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:27.327157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14c264-0333-41f3-a21f-a252a7d5b1af None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:27.330150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54aae6-0a87-416e-8aa8-8d88e364f7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:27.331039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54aae6-0a87-416e-8aa8-8d88e364f7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:27.331314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54aae6-0a87-416e-8aa8-8d88e364f7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:27.331578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54aae6-0a87-416e-8aa8-8d88e364f7c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:27.343528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:27.352873 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54aae6-0a87-416e-8aa8-8d88e364f7c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:27.354394 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54aae6-0a87-416e-8aa8-8d88e364f7c5 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:27.374095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-3b54aae6-0a87-416e-8aa8-8d88e364f7c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:27.374095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:27.374095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:27.374095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:27.374095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:27.374095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:27.374095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:27.374095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:27.374095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:27.374095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:27.374095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:27.374095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:27.374095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:27.374095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:27.374095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:27.374095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:27.375196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:27.375196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:27.375196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:27.375196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:27.375196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:27.375196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:27.375196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:27.375196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:27.375196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:27.375196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:27.375196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:27.375196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:27.375196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:27.375196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:27.375196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:27.375196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:27.375196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:27.377261 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:27.377261 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:27.377261 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:29:27.377261 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 9/18] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:29:27.384637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54aae6-0a87-416e-8aa8-8d88e364f7c5 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:27.450948 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:27.467322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14c264-0333-41f3-a21f-a252a7d5b1af None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:27.505888 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c6e2a-c885-48ad-8f0a-beef0b5ff7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:27.506538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c6e2a-c885-48ad-8f0a-beef0b5ff7f1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:27.506816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c6e2a-c885-48ad-8f0a-beef0b5ff7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:27.507085 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c6e2a-c885-48ad-8f0a-beef0b5ff7f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:27.508382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c6e2a-c885-48ad-8f0a-beef0b5ff7f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:27.509775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c6e2a-c885-48ad-8f0a-beef0b5ff7f1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:27.511381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 10/19] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:27.517789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c6e2a-c885-48ad-8f0a-beef0b5ff7f1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:27.519830 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54aae6-0a87-416e-8aa8-8d88e364f7c5 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:27.572066 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac2a91-e1b9-42b3-9cfd-cd071275de25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:27.573070 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac2a91-e1b9-42b3-9cfd-cd071275de25 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:27.573558 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac2a91-e1b9-42b3-9cfd-cd071275de25 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:27.574031 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac2a91-e1b9-42b3-9cfd-cd071275de25 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:27.575212 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac2a91-e1b9-42b3-9cfd-cd071275de25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:27.575585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac2a91-e1b9-42b3-9cfd-cd071275de25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:27.578324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac2a91-e1b9-42b3-9cfd-cd071275de25 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:27.603936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 10/20] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:27.604162 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:27.638944 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c6e2a-c885-48ad-8f0a-beef0b5ff7f1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:27.701790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64003e09-1dfb-4fc3-a4b0-fd2ede1905bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:27.702453 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64003e09-1dfb-4fc3-a4b0-fd2ede1905bb None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:27.702733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64003e09-1dfb-4fc3-a4b0-fd2ede1905bb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:27.703028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64003e09-1dfb-4fc3-a4b0-fd2ede1905bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:27.703802 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64003e09-1dfb-4fc3-a4b0-fd2ede1905bb None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.domain.id=default, user.id=5f1827dde46f4915bfe3277a281b43bd) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:27.704149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64003e09-1dfb-4fc3-a4b0-fd2ede1905bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:27.705942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64003e09-1dfb-4fc3-a4b0-fd2ede1905bb None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:27.715094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 11/21] 10.222.0.25 () {56 vars in 1322 bytes} [Tue Aug 30 14:29:27 2022] GET /identity/v3/role_assignments?user.id=5f1827dde46f4915bfe3277a281b43bd&scope.domain.id=default&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 233 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:29.976697 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6513f8-dc7e-4dca-8a23-2c1c75fb1313 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:29.977710 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6513f8-dc7e-4dca-8a23-2c1c75fb1313 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:29.978189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6513f8-dc7e-4dca-8a23-2c1c75fb1313 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:29.979799 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 11/22] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:29.985681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3429c24-7991-46b2-bd0f-a20f6911f674 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:29.986648 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3429c24-7991-46b2-bd0f-a20f6911f674 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:29.987117 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3429c24-7991-46b2-bd0f-a20f6911f674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:30.006728 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f3429c24-7991-46b2-bd0f-a20f6911f674 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:30.008373 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f3429c24-7991-46b2-bd0f-a20f6911f674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:30.099491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 12/23] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:30.239310 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7041863-fd38-4125-a865-af05521a52ae None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:30.239950 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7041863-fd38-4125-a865-af05521a52ae None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:30.240449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7041863-fd38-4125-a865-af05521a52ae None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:30.242079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 12/24] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:30.248169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02138e43-e6d6-4ee7-8ad2-096632a4f288 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:30.248816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02138e43-e6d6-4ee7-8ad2-096632a4f288 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:30.249301 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02138e43-e6d6-4ee7-8ad2-096632a4f288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:30.269180 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-02138e43-e6d6-4ee7-8ad2-096632a4f288 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:30.270747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-02138e43-e6d6-4ee7-8ad2-096632a4f288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:30.284077 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3af1f-c70f-4e92-8cf5-cf99ff626d2a None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:30.284742 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3af1f-c70f-4e92-8cf5-cf99ff626d2a None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:30.285210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3af1f-c70f-4e92-8cf5-cf99ff626d2a None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:30.286829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 13/25] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:30.293172 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f0dc0-4cae-4845-8b62-0aac1c86465d None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:30.293790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f0dc0-4cae-4845-8b62-0aac1c86465d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:30.294273 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f0dc0-4cae-4845-8b62-0aac1c86465d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:30.323610 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-346f0dc0-4cae-4845-8b62-0aac1c86465d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:30.326070 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-346f0dc0-4cae-4845-8b62-0aac1c86465d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:30.385318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 13/26] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:30.457057 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 14/27] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:30.481852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02138e43-e6d6-4ee7-8ad2-096632a4f288 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:30.486328 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:30.624747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02138e43-e6d6-4ee7-8ad2-096632a4f288 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:30.677856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164188ee-c0ce-43cd-8934-d2ae438e2660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:30.678615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164188ee-c0ce-43cd-8934-d2ae438e2660 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:30.678901 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164188ee-c0ce-43cd-8934-d2ae438e2660 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:30.679173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164188ee-c0ce-43cd-8934-d2ae438e2660 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:30.694727 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164188ee-c0ce-43cd-8934-d2ae438e2660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:30.696275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164188ee-c0ce-43cd-8934-d2ae438e2660 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:30.721758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-164188ee-c0ce-43cd-8934-d2ae438e2660 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:30.721758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:30.721758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:30.721758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:30.721758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:30.721758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:30.721758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:30.721758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:30.721758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:30.721758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:30.721758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:30.721758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:30.721758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:30.721758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:30.721758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:30.721758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:30.722663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:30.722663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:30.722663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:30.722663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:30.722663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:30.722663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:30.722663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:30.722663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:30.722663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:30.722663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:30.722663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:30.722663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:30.722663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:30.722663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:30.722663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:30.722663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:30.722663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:30.724165 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:30.724165 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:30.724165 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:29:30.724165 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 14/28] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:30.728930 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f0dc0-4cae-4845-8b62-0aac1c86465d None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:30.807708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:30.872582 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164188ee-c0ce-43cd-8934-d2ae438e2660 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:30.949390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f0dc0-4cae-4845-8b62-0aac1c86465d None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:30.949565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:31.036451 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164188ee-c0ce-43cd-8934-d2ae438e2660 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:31.043984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940df0a-fc92-4343-b4d5-6a3ece1038a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:31.044677 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940df0a-fc92-4343-b4d5-6a3ece1038a8 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:31.044924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940df0a-fc92-4343-b4d5-6a3ece1038a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:31.045188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940df0a-fc92-4343-b4d5-6a3ece1038a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:31.045855 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940df0a-fc92-4343-b4d5-6a3ece1038a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:31.046153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940df0a-fc92-4343-b4d5-6a3ece1038a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:31.047564 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940df0a-fc92-4343-b4d5-6a3ece1038a8 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:31.080454 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 15/29] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:31.096223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940df0a-fc92-4343-b4d5-6a3ece1038a8 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:31.110300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc4329-b685-44c5-b4b8-bade49f312e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:31.111409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc4329-b685-44c5-b4b8-bade49f312e1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:31.111887 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc4329-b685-44c5-b4b8-bade49f312e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:31.112638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc4329-b685-44c5-b4b8-bade49f312e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:31.149237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc4329-b685-44c5-b4b8-bade49f312e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:31.165501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-e9fc4329-b685-44c5-b4b8-bade49f312e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:31.165501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:31.165501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:31.165501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:31.165501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:31.165501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:31.165501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:31.165501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:31.165501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:31.165501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:31.165501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:31.165501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:31.165501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:31.165501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:31.165501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:31.165501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:31.166428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:31.166428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:31.166428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:31.166428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:31.166428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:31.166428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:31.166428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:31.166428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:31.166428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:31.166428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:31.166428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:31.166428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:31.166428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:31.166428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:31.166428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:31.166428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:31.167270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:31.167270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:31.167270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:31.167270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:31.167270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:31.167270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:31.167270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:31.167270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:31.167270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:31.167270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:31.167270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:31.167270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:29:31.174974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 15/30] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:29:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:31.178219 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1f3d5-eb04-4a5f-8514-f29e91cd3bbd None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:31.178955 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1f3d5-eb04-4a5f-8514-f29e91cd3bbd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:31.180404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1f3d5-eb04-4a5f-8514-f29e91cd3bbd None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:31.186426 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 16/31] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:31.186426 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1f3d5-eb04-4a5f-8514-f29e91cd3bbd None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:31.264673 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940df0a-fc92-4343-b4d5-6a3ece1038a8 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:31.273208 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:31.306629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c559e9-f835-4ad5-88dc-907308dcbf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:31.307275 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c559e9-f835-4ad5-88dc-907308dcbf27 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:31.307555 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c559e9-f835-4ad5-88dc-907308dcbf27 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:31.307827 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c559e9-f835-4ad5-88dc-907308dcbf27 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:31.308871 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c559e9-f835-4ad5-88dc-907308dcbf27 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:31.310430 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c559e9-f835-4ad5-88dc-907308dcbf27 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:31.358091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1f3d5-eb04-4a5f-8514-f29e91cd3bbd None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:31.392603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:31.428527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9b5b3-cf2b-4312-ad52-99b68dc595c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:31.429937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9b5b3-cf2b-4312-ad52-99b68dc595c0 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:31.430643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9b5b3-cf2b-4312-ad52-99b68dc595c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:31.431302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9b5b3-cf2b-4312-ad52-99b68dc595c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:31.432654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff9b5b3-cf2b-4312-ad52-99b68dc595c0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:31.435268 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff9b5b3-cf2b-4312-ad52-99b68dc595c0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:31.464542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 16/32] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Aug 30 14:29:31 2022] POST /identity/v3/projects => generated 301 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:31.472529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c559e9-f835-4ad5-88dc-907308dcbf27 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:31.647859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c559e9-f835-4ad5-88dc-907308dcbf27 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:31.654789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 17/33] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Aug 30 14:29:31 2022] POST /identity/v3/roles => generated 219 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:31.656877 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5f04f-a106-4f6f-a463-570ddad11d0e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:31.657407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5f04f-a106-4f6f-a463-570ddad11d0e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:31.657998 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5f04f-a106-4f6f-a463-570ddad11d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:31.691138 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7bc5f04f-a106-4f6f-a463-570ddad11d0e None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:31.693692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7bc5f04f-a106-4f6f-a463-570ddad11d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:31.708392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4b573-9f47-492d-9a6e-4c10982234ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:31.710449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4b573-9f47-492d-9a6e-4c10982234ac None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:31.710590 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4b573-9f47-492d-9a6e-4c10982234ac None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:31.711127 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4b573-9f47-492d-9a6e-4c10982234ac None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:31.712933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e4b573-9f47-492d-9a6e-4c10982234ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:31.713327 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e4b573-9f47-492d-9a6e-4c10982234ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:31.716347 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e4b573-9f47-492d-9a6e-4c10982234ac None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:31.745379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 17/34] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:31.753008 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afec722-c860-4e1f-8e2e-523791bf0b4e None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:31.753600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afec722-c860-4e1f-8e2e-523791bf0b4e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:31.754099 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afec722-c860-4e1f-8e2e-523791bf0b4e None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:31.760241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 18/35] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:31 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:31.761716 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afec722-c860-4e1f-8e2e-523791bf0b4e None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:31.793690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 18/36] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:31.795655 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c1a23-45db-4054-b4d2-c5a1d14c1100 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:31.796186 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c1a23-45db-4054-b4d2-c5a1d14c1100 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:31.796625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c1a23-45db-4054-b4d2-c5a1d14c1100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:31.820097 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:31.836676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-846c1a23-45db-4054-b4d2-c5a1d14c1100 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:31.841080 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-846c1a23-45db-4054-b4d2-c5a1d14c1100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:31.941288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 19/37] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:32.025783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afec722-c860-4e1f-8e2e-523791bf0b4e None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:32.091320 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929297e-1bd2-4606-b218-f3571f227b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:32.092414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929297e-1bd2-4606-b218-f3571f227b66 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:32.092899 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929297e-1bd2-4606-b218-f3571f227b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:32.093368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929297e-1bd2-4606-b218-f3571f227b66 None admin] PATH_INFO: `/v3/users/5f1827dde46f4915bfe3277a281b43bd` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:32.096092 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929297e-1bd2-4606-b218-f3571f227b66 None admin] RBAC: Authorizing `identity:get_user(user_id=5f1827dde46f4915bfe3277a281b43bd)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:32.098866 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929297e-1bd2-4606-b218-f3571f227b66 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:32.102134 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 19/38] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:31 2022] GET /identity/v3/users/5f1827dde46f4915bfe3277a281b43bd => generated 247 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:32.108896 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c1a23-45db-4054-b4d2-c5a1d14c1100 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:32.154999 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:32.223024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cbfb9-4047-4156-b540-62b40bd6e785 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:32.223925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cbfb9-4047-4156-b540-62b40bd6e785 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:32.224654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cbfb9-4047-4156-b540-62b40bd6e785 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:32.227324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 20/39] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:32.232046 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c1a23-45db-4054-b4d2-c5a1d14c1100 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:32.234929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e3b9e-05c4-48b9-9a1c-9d0dd9bf0aac None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:32.235700 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e3b9e-05c4-48b9-9a1c-9d0dd9bf0aac None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:32.236480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e3b9e-05c4-48b9-9a1c-9d0dd9bf0aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:32.269735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-518e3b9e-05c4-48b9-9a1c-9d0dd9bf0aac None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:32.273039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-518e3b9e-05c4-48b9-9a1c-9d0dd9bf0aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:32.389502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45786555-86e1-42b7-b870-a080e978e1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:32.390420 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45786555-86e1-42b7-b870-a080e978e1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:32.390852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45786555-86e1-42b7-b870-a080e978e1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:32.391250 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45786555-86e1-42b7-b870-a080e978e1b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:32.392918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45786555-86e1-42b7-b870-a080e978e1b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:32.394100 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 21/40] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:32.395044 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45786555-86e1-42b7-b870-a080e978e1b2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:32.397105 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 20/41] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:32.401601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e3b9e-05c4-48b9-9a1c-9d0dd9bf0aac None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:32.404527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45786555-86e1-42b7-b870-a080e978e1b2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:32.436923 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:32.539164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45786555-86e1-42b7-b870-a080e978e1b2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:32.539592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e3b9e-05c4-48b9-9a1c-9d0dd9bf0aac None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:32.547843 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:32.580122 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d490d98-114a-476f-adc4-d3c55d6ac502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:32.580804 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d490d98-114a-476f-adc4-d3c55d6ac502 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:32.581047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d490d98-114a-476f-adc4-d3c55d6ac502 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:32.583793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d490d98-114a-476f-adc4-d3c55d6ac502 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:32.583793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d490d98-114a-476f-adc4-d3c55d6ac502 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:32.583793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d490d98-114a-476f-adc4-d3c55d6ac502 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:32.598324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123758f-5b67-43d5-909e-4bfb5a73a695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:32.599605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123758f-5b67-43d5-909e-4bfb5a73a695 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:32.600185 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123758f-5b67-43d5-909e-4bfb5a73a695 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:32.600752 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123758f-5b67-43d5-909e-4bfb5a73a695 None admin] PATH_INFO: `/v3/domains/default/users/5f1827dde46f4915bfe3277a281b43bd/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:32.604261 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123758f-5b67-43d5-909e-4bfb5a73a695 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5f1827dde46f4915bfe3277a281b43bd, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:32.606076 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 22/42] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Aug 30 14:29:32 2022] POST /identity/v3/roles => generated 225 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:32.607159 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123758f-5b67-43d5-909e-4bfb5a73a695 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:32.607258 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d490d98-114a-476f-adc4-d3c55d6ac502 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:32.646039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 21/43] 10.222.0.25 () {58 vars in 1361 bytes} [Tue Aug 30 14:29:32 2022] PUT /identity/v3/domains/default/users/5f1827dde46f4915bfe3277a281b43bd/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:29:32.687592 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123758f-5b67-43d5-909e-4bfb5a73a695 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:32.765300 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:32.797975 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d490d98-114a-476f-adc4-d3c55d6ac502 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:32.869707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c39e1f-2320-450d-894d-e660115bbc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:32.871214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c39e1f-2320-450d-894d-e660115bbc92 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:32.871947 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c39e1f-2320-450d-894d-e660115bbc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:32.872683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c39e1f-2320-450d-894d-e660115bbc92 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:32.874040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c39e1f-2320-450d-894d-e660115bbc92 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:32.876985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c39e1f-2320-450d-894d-e660115bbc92 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:32.880238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123758f-5b67-43d5-909e-4bfb5a73a695 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:32.907614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 23/44] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Aug 30 14:29:32 2022] POST /identity/v3/roles => generated 223 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:32.938351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541a23d-cec8-4d9d-8395-01c416c5a683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:32.940801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541a23d-cec8-4d9d-8395-01c416c5a683 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:32.941312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541a23d-cec8-4d9d-8395-01c416c5a683 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:32.941895 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541a23d-cec8-4d9d-8395-01c416c5a683 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:32.944178 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9541a23d-cec8-4d9d-8395-01c416c5a683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:32.946966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9541a23d-cec8-4d9d-8395-01c416c5a683 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:32.950628 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 22/45] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:32.959287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c39e1f-2320-450d-894d-e660115bbc92 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:33.091647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c39e1f-2320-450d-894d-e660115bbc92 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:33.134773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e1f77-6df3-4873-ac10-6094b635d5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:33.136147 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e1f77-6df3-4873-ac10-6094b635d5d9 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:33.136605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e1f77-6df3-4873-ac10-6094b635d5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:33.136957 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e1f77-6df3-4873-ac10-6094b635d5d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:33.137752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e1f77-6df3-4873-ac10-6094b635d5d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:33.139339 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e1f77-6df3-4873-ac10-6094b635d5d9 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:33.153460 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:33.172218 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 24/46] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Aug 30 14:29:32 2022] POST /identity/v3/projects => generated 312 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:34.902308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5fce3-45c5-4fe6-aa0e-c0f3db388522 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:34.902873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5fce3-45c5-4fe6-aa0e-c0f3db388522 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:34.902873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5fce3-45c5-4fe6-aa0e-c0f3db388522 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:34.904194 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 23/47] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:34.910078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b89f7d-0519-44af-8388-e500fa5c5d99 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:34.910642 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b89f7d-0519-44af-8388-e500fa5c5d99 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:34.910642 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b89f7d-0519-44af-8388-e500fa5c5d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:34.931036 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-f3b89f7d-0519-44af-8388-e500fa5c5d99 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:34.932292 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f3b89f7d-0519-44af-8388-e500fa5c5d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:35.006888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 25/48] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:35.277792 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5fce3-45c5-4fe6-aa0e-c0f3db388522 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:35.316445 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:35.411483 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5fce3-45c5-4fe6-aa0e-c0f3db388522 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:35.476282 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9604bf-86d6-4941-8514-d65fb1eec343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:35.477399 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9604bf-86d6-4941-8514-d65fb1eec343 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:35.477911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9604bf-86d6-4941-8514-d65fb1eec343 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:35.478418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9604bf-86d6-4941-8514-d65fb1eec343 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:35.502034 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9604bf-86d6-4941-8514-d65fb1eec343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:35.510772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-6f9604bf-86d6-4941-8514-d65fb1eec343 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:35.510772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:35.510772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:35.510772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:35.510772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:35.510772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:35.510772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:35.510772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:35.510772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:35.510772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:35.510772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:35.510772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:35.510772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:35.510772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:35.510772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:35.510772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:35.512481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:35.512481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:35.512481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:35.512481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:35.512481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:35.512481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:35.512481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:35.512481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:35.512481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:35.512481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:35.512481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:35.512481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:35.512481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:35.512481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:35.512481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:35.512481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:35.513922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:35.513922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:35.513922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:35.513922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:35.513922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:35.513922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:35.513922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:35.513922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:35.513922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:35.513922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:35.513922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:35.513922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:29:35.513922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 24/49] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:29:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:35.522758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b89f7d-0519-44af-8388-e500fa5c5d99 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:35.593949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94f9ed-b18e-4844-8493-c0edaf763764 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:35.594484 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94f9ed-b18e-4844-8493-c0edaf763764 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:35.595092 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94f9ed-b18e-4844-8493-c0edaf763764 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:35.597415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 25/50] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:35.606259 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b14d-ec1a-4346-a980-e07138ac306e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:35.606765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b14d-ec1a-4346-a980-e07138ac306e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:35.607249 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b14d-ec1a-4346-a980-e07138ac306e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:35.638167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-3964b14d-ec1a-4346-a980-e07138ac306e None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:35.640335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3964b14d-ec1a-4346-a980-e07138ac306e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:35.660150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b89f7d-0519-44af-8388-e500fa5c5d99 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:35.686111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:35.707957 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce45545-9a1c-4d77-83b1-a744b1507d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:35.708973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce45545-9a1c-4d77-83b1-a744b1507d86 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:35.709435 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce45545-9a1c-4d77-83b1-a744b1507d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:35.710210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce45545-9a1c-4d77-83b1-a744b1507d86 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:35.711353 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce45545-9a1c-4d77-83b1-a744b1507d86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:35.711853 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce45545-9a1c-4d77-83b1-a744b1507d86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:35.714317 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce45545-9a1c-4d77-83b1-a744b1507d86 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:35.725572 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 26/51] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:35.732299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce45545-9a1c-4d77-83b1-a744b1507d86 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:35.750189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 26/52] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:35.853371 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce45545-9a1c-4d77-83b1-a744b1507d86 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:35.896755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33d321-bcd6-4cec-812f-672ca6f1a09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:35.897409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33d321-bcd6-4cec-812f-672ca6f1a09a None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:35.897685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33d321-bcd6-4cec-812f-672ca6f1a09a None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:35.897959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33d321-bcd6-4cec-812f-672ca6f1a09a None admin] PATH_INFO: `/v3/users/5f1827dde46f4915bfe3277a281b43bd` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:35.899145 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33d321-bcd6-4cec-812f-672ca6f1a09a None admin] RBAC: Authorizing `identity:get_user(user_id=5f1827dde46f4915bfe3277a281b43bd)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:35.900676 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33d321-bcd6-4cec-812f-672ca6f1a09a None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:35.902313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 27/53] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:35 2022] GET /identity/v3/users/5f1827dde46f4915bfe3277a281b43bd => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:35.908494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964b14d-ec1a-4346-a980-e07138ac306e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:35.938838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:36.030280 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964b14d-ec1a-4346-a980-e07138ac306e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:36.050929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33d321-bcd6-4cec-812f-672ca6f1a09a None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:36.073563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbc6b6-990b-4721-87c5-d50e3d5a007f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:36.074563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbc6b6-990b-4721-87c5-d50e3d5a007f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:36.075008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbc6b6-990b-4721-87c5-d50e3d5a007f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:36.075447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbc6b6-990b-4721-87c5-d50e3d5a007f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:36.077331 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbc6b6-990b-4721-87c5-d50e3d5a007f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:36.080054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbc6b6-990b-4721-87c5-d50e3d5a007f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:36.082816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 27/54] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:36.092708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbc6b6-990b-4721-87c5-d50e3d5a007f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:36.128379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:36.192781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33d321-bcd6-4cec-812f-672ca6f1a09a None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:36.215687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:36.229564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbc6b6-990b-4721-87c5-d50e3d5a007f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:36.238436 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8986a4a-e4e7-42c4-9dda-e48004512a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:36.239242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8986a4a-e4e7-42c4-9dda-e48004512a9c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:36.239585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8986a4a-e4e7-42c4-9dda-e48004512a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:36.239919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8986a4a-e4e7-42c4-9dda-e48004512a9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:36.241439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8986a4a-e4e7-42c4-9dda-e48004512a9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:36.243187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8986a4a-e4e7-42c4-9dda-e48004512a9c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:36.245143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 28/55] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:36.251708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8986a4a-e4e7-42c4-9dda-e48004512a9c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:36.273077 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6f8ce-9b0b-4705-9a21-9f3f20bd021d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:36.273954 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6f8ce-9b0b-4705-9a21-9f3f20bd021d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:36.274321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6f8ce-9b0b-4705-9a21-9f3f20bd021d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:36.274659 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6f8ce-9b0b-4705-9a21-9f3f20bd021d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:36.275705 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6f8ce-9b0b-4705-9a21-9f3f20bd021d None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.domain.id=default, user.id=5f1827dde46f4915bfe3277a281b43bd) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:36.276145 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6f8ce-9b0b-4705-9a21-9f3f20bd021d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:36.277818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6f8ce-9b0b-4705-9a21-9f3f20bd021d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:36.286188 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 28/56] 10.222.0.25 () {56 vars in 1322 bytes} [Tue Aug 30 14:29:36 2022] GET /identity/v3/role_assignments?user.id=5f1827dde46f4915bfe3277a281b43bd&scope.domain.id=default&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 530 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:36.382702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8986a4a-e4e7-42c4-9dda-e48004512a9c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:36.403960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:36.432282 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b92dc-24ce-42a0-874e-cacf75b5cf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:36.433243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b92dc-24ce-42a0-874e-cacf75b5cf28 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:36.433676 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b92dc-24ce-42a0-874e-cacf75b5cf28 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:36.434105 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b92dc-24ce-42a0-874e-cacf75b5cf28 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:36.435166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b92dc-24ce-42a0-874e-cacf75b5cf28 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:36.437826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b92dc-24ce-42a0-874e-cacf75b5cf28 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:36.479843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 29/57] 10.222.0.25 () {60 vars in 1140 bytes} [Tue Aug 30 14:29:36 2022] POST /identity/v3/projects => generated 298 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:29:38.487245 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d1b97-54e6-4c49-82df-05be7c14fc4f None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:38.487831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d1b97-54e6-4c49-82df-05be7c14fc4f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:38.487831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d1b97-54e6-4c49-82df-05be7c14fc4f None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:38.489225 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 29/58] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:38.495084 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a375bb9-9588-416f-99b0-86b5e28c345f None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:38.495590 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a375bb9-9588-416f-99b0-86b5e28c345f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:38.495590 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a375bb9-9588-416f-99b0-86b5e28c345f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:38.515521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-0a375bb9-9588-416f-99b0-86b5e28c345f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:38.516791 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0a375bb9-9588-416f-99b0-86b5e28c345f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:38.603847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 30/59] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:38.892104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d1b97-54e6-4c49-82df-05be7c14fc4f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:39.010789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d1b97-54e6-4c49-82df-05be7c14fc4f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:39.048959 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c106f9-3df9-4f0e-90b2-6d9744c79141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:39.049687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c106f9-3df9-4f0e-90b2-6d9744c79141 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:39.049965 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c106f9-3df9-4f0e-90b2-6d9744c79141 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:39.050241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c106f9-3df9-4f0e-90b2-6d9744c79141 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:39.051444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c106f9-3df9-4f0e-90b2-6d9744c79141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:39.052939 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c106f9-3df9-4f0e-90b2-6d9744c79141 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:39.054568 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 30/60] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:39.060610 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a375bb9-9588-416f-99b0-86b5e28c345f None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:39.167133 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:39.170563 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a375bb9-9588-416f-99b0-86b5e28c345f None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:39.207776 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1206d5a-8f38-42e9-a2be-4cd653e0800b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:39.208441 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1206d5a-8f38-42e9-a2be-4cd653e0800b None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:39.208720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1206d5a-8f38-42e9-a2be-4cd653e0800b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:39.208989 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1206d5a-8f38-42e9-a2be-4cd653e0800b None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:39.210045 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1206d5a-8f38-42e9-a2be-4cd653e0800b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:39.211508 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1206d5a-8f38-42e9-a2be-4cd653e0800b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:39.242648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 31/61] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:29:39 2022] POST /identity/v3/users => generated 275 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:42.940131 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f6510-2331-4cb6-b937-73e5366e3fd1 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:42.940988 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f6510-2331-4cb6-b937-73e5366e3fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:42.941400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f6510-2331-4cb6-b937-73e5366e3fd1 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:42.943287 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 31/62] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:42.949307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197a99e-e116-4732-9b32-9dd76168a840 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:42.950149 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197a99e-e116-4732-9b32-9dd76168a840 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:42.950600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197a99e-e116-4732-9b32-9dd76168a840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:42.978540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-0197a99e-e116-4732-9b32-9dd76168a840 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:42.979990 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0197a99e-e116-4732-9b32-9dd76168a840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:43.080112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 32/63] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:43.150660 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04382d23-b154-489e-913d-65ffad6eacd8 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.151314 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04382d23-b154-489e-913d-65ffad6eacd8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.151773 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04382d23-b154-489e-913d-65ffad6eacd8 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.153701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 32/64] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:43.159140 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91679b42-2c42-44c1-bb24-edcbc6e60a66 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.159748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91679b42-2c42-44c1-bb24-edcbc6e60a66 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.160255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91679b42-2c42-44c1-bb24-edcbc6e60a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.184934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-91679b42-2c42-44c1-bb24-edcbc6e60a66 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:43.186484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-91679b42-2c42-44c1-bb24-edcbc6e60a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:43.276270 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 33/65] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:43.520123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec4f9f-46da-498d-a071-2cf32056c22a None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.520820 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec4f9f-46da-498d-a071-2cf32056c22a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.521297 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec4f9f-46da-498d-a071-2cf32056c22a None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.522855 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 33/66] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:43.528755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755c59f-2dc2-43e5-af7c-e748a0d1362b None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.529320 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755c59f-2dc2-43e5-af7c-e748a0d1362b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.529783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755c59f-2dc2-43e5-af7c-e748a0d1362b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.539725 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec4f9f-46da-498d-a071-2cf32056c22a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:43.557109 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-4755c59f-2dc2-43e5-af7c-e748a0d1362b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:43.559344 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4755c59f-2dc2-43e5-af7c-e748a0d1362b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:43.583626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:43.594536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:43.674788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 34/67] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:43.681608 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4132a38-4dd9-43f2-9251-26ddc17bb6b7 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.682155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4132a38-4dd9-43f2-9251-26ddc17bb6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.682864 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4132a38-4dd9-43f2-9251-26ddc17bb6b7 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.684945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 35/68] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:43 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:43.686654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295f6db-9ccc-4f9e-b55b-444a9febaf6f None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.687138 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295f6db-9ccc-4f9e-b55b-444a9febaf6f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.687596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295f6db-9ccc-4f9e-b55b-444a9febaf6f None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.689424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 36/69] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:43.696788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7962df-4a31-4ab9-bf0f-2b0935205fea None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.698003 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7962df-4a31-4ab9-bf0f-2b0935205fea None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.698537 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7962df-4a31-4ab9-bf0f-2b0935205fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.721287 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec4f9f-46da-498d-a071-2cf32056c22a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:43.792664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75bc0f-ac4c-42f8-82c4-9c1ae70d9faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:43.793639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75bc0f-ac4c-42f8-82c4-9c1ae70d9faf None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.794093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75bc0f-ac4c-42f8-82c4-9c1ae70d9faf None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.794541 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75bc0f-ac4c-42f8-82c4-9c1ae70d9faf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.817717 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b75bc0f-ac4c-42f8-82c4-9c1ae70d9faf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:43.827706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-9b75bc0f-ac4c-42f8-82c4-9c1ae70d9faf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:43.827706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:43.827706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:43.827706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:43.827706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:43.827706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:43.827706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:43.827706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:43.827706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:43.827706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:43.827706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:43.827706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:43.827706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:43.827706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:43.827706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:43.827706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:43.829527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:43.829527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:43.829527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:43.829527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:43.829527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:43.829527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:43.829527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:43.829527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:43.829527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:43.829527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:43.829527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:43.829527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:43.829527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:43.829527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:43.829527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:43.829527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:43.830867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:43.830867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:43.830867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:43.830867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:43.830867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:43.830867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:43.830867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:43.830867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:43.830867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:43.830867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:43.830867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:43.830867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:29:43.830867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 34/70] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:43.832789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75bc0f-ac4c-42f8-82c4-9c1ae70d9faf None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:43.835248 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-5c7962df-4a31-4ab9-bf0f-2b0935205fea None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:43.839325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5c7962df-4a31-4ab9-bf0f-2b0935205fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:43.939387 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 37/71] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:43.941620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e21d2-98d3-4c1f-9f81-0e61a5ab73ea None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.942179 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e21d2-98d3-4c1f-9f81-0e61a5ab73ea None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.942665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e21d2-98d3-4c1f-9f81-0e61a5ab73ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.970003 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:43.970840 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-2e6e21d2-98d3-4c1f-9f81-0e61a5ab73ea None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:43.972048 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2e6e21d2-98d3-4c1f-9f81-0e61a5ab73ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:44.006756 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75bc0f-ac4c-42f8-82c4-9c1ae70d9faf None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:44.074434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86102b2-c58d-4ebd-a47d-a494dda6ec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:44.075443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86102b2-c58d-4ebd-a47d-a494dda6ec6d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:44.075921 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86102b2-c58d-4ebd-a47d-a494dda6ec6d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:44.076413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 38/72] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:44.077081 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86102b2-c58d-4ebd-a47d-a494dda6ec6d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:44.077963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e21d2-98d3-4c1f-9f81-0e61a5ab73ea None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:44.097676 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:44.114849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86102b2-c58d-4ebd-a47d-a494dda6ec6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:44.125502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-c86102b2-c58d-4ebd-a47d-a494dda6ec6d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:44.125502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:44.125502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:44.125502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:44.125502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:44.125502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:44.125502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:44.125502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:44.125502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:44.125502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:44.125502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:44.125502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:44.125502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:44.125502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:44.125502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:44.125502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:44.127352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:44.127352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:44.127352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:44.127352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:44.127352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:44.127352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:44.127352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:44.127352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:44.127352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:44.127352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:44.127352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:44.127352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:44.127352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:44.127352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:44.127352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:44.127352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:44.128825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:44.128825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:44.128825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:44.128825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:44.128825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:44.128825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:44.128825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:44.128825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:44.128825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:44.128825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:44.128825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:44.128825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:29:44.128825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 35/73] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:29:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:44.130014 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86102b2-c58d-4ebd-a47d-a494dda6ec6d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:44.299506 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e21d2-98d3-4c1f-9f81-0e61a5ab73ea None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:44.330882 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86102b2-c58d-4ebd-a47d-a494dda6ec6d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:44.362095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb77f57-dafd-4de1-bd7b-7bbd35872b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:44.363695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb77f57-dafd-4de1-bd7b-7bbd35872b24 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:44.364428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb77f57-dafd-4de1-bd7b-7bbd35872b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:44.365121 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb77f57-dafd-4de1-bd7b-7bbd35872b24 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:44.366489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb77f57-dafd-4de1-bd7b-7bbd35872b24 None admin] RBAC: Adding query filter params (name=member) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:44.367248 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb77f57-dafd-4de1-bd7b-7bbd35872b24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:44.370145 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb77f57-dafd-4de1-bd7b-7bbd35872b24 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:44.389465 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 39/74] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:44.390922 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb77f57-dafd-4de1-bd7b-7bbd35872b24 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:44.401243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c63a51-11ca-41c9-94f8-669b7e0e04a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:44.402304 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c63a51-11ca-41c9-94f8-669b7e0e04a2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:44.402715 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c63a51-11ca-41c9-94f8-669b7e0e04a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:44.403156 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c63a51-11ca-41c9-94f8-669b7e0e04a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:44.420896 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c63a51-11ca-41c9-94f8-669b7e0e04a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:44.430402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-98c63a51-11ca-41c9-94f8-669b7e0e04a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:44.430402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:44.430402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:44.430402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:44.430402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:44.430402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:44.430402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:44.430402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:44.430402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:44.430402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:44.430402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:44.430402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:44.430402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:44.430402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:44.430402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:44.430402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:44.431408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:44.431408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:44.431408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:44.431408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:44.431408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:44.431408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:44.431408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:44.431408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:44.431408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:44.431408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:44.431408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:44.431408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:44.431408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:44.431408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:44.431408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:44.431408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:44.432251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:44.432251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:44.432251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:44.432251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:44.432251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:44.432251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:44.432251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:44.432251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:44.432251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:44.432251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:44.432251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:44.432251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:29:44.433435 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 36/75] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:44.434538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c63a51-11ca-41c9-94f8-669b7e0e04a2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:44.437817 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:44.511170 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:44.573100 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb77f57-dafd-4de1-bd7b-7bbd35872b24 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:44.616335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e8361-9515-4c77-b1fa-ef7c5e72c44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:44.616890 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c63a51-11ca-41c9-94f8-669b7e0e04a2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:44.617023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e8361-9515-4c77-b1fa-ef7c5e72c44b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:44.617227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e8361-9515-4c77-b1fa-ef7c5e72c44b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:44.617495 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e8361-9515-4c77-b1fa-ef7c5e72c44b None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:44.618181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e8361-9515-4c77-b1fa-ef7c5e72c44b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:44.618484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e8361-9515-4c77-b1fa-ef7c5e72c44b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:44.619881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e8361-9515-4c77-b1fa-ef7c5e72c44b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:44.637671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 40/76] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:44.637671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e8361-9515-4c77-b1fa-ef7c5e72c44b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:44.669874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee916a-7773-4276-a5bd-1fcbbf314355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:44.670649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee916a-7773-4276-a5bd-1fcbbf314355 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:44.671012 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee916a-7773-4276-a5bd-1fcbbf314355 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:44.671385 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee916a-7773-4276-a5bd-1fcbbf314355 None admin] PATH_INFO: `/v3/users/7ded74de77c442b1853b2349fb3567ad` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:44.688160 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee916a-7773-4276-a5bd-1fcbbf314355 None admin] RBAC: Authorizing `identity:get_user(user_id=7ded74de77c442b1853b2349fb3567ad)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:44.690784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee916a-7773-4276-a5bd-1fcbbf314355 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:44.692924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 37/77] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:44 2022] GET /identity/v3/users/7ded74de77c442b1853b2349fb3567ad => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:44.694263 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee916a-7773-4276-a5bd-1fcbbf314355 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:44.770248 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e8361-9515-4c77-b1fa-ef7c5e72c44b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:44.771895 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:44.803169 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:44.811527 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c0098-412f-4e99-a4af-ab0cdbd2d041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:44.812443 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c0098-412f-4e99-a4af-ab0cdbd2d041 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:44.812892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c0098-412f-4e99-a4af-ab0cdbd2d041 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:44.813331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c0098-412f-4e99-a4af-ab0cdbd2d041 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:44.831300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee916a-7773-4276-a5bd-1fcbbf314355 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:44.834774 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c0098-412f-4e99-a4af-ab0cdbd2d041 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:44.845626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-9e7c0098-412f-4e99-a4af-ab0cdbd2d041 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:44.845626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:44.845626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:44.845626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:44.845626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:44.845626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:44.845626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:44.845626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:44.845626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:44.845626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:44.845626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:44.845626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:44.845626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:44.845626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:44.845626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:44.845626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:44.847432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:44.847432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:44.847432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:44.847432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:44.847432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:44.847432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:44.847432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:44.847432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:44.847432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:44.847432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:44.847432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:44.847432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:44.847432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:44.847432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:44.847432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:44.847432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:44.854039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:44.854039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:44.854039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:44.854039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:44.854039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:44.854039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:44.854039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:44.854039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:44.854039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:44.854039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:44.854039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:44.854039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:29:44.854039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 41/78] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:29:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:44.854039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c0098-412f-4e99-a4af-ab0cdbd2d041 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:44.880345 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a1390-3674-4104-bf1a-08db91135986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:44.881044 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a1390-3674-4104-bf1a-08db91135986 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:44.881433 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a1390-3674-4104-bf1a-08db91135986 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:44.881830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a1390-3674-4104-bf1a-08db91135986 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:44.882670 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a1390-3674-4104-bf1a-08db91135986 None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:44.883085 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a1390-3674-4104-bf1a-08db91135986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:44.884719 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a1390-3674-4104-bf1a-08db91135986 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:44.895356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 38/79] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:44.896830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a1390-3674-4104-bf1a-08db91135986 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:44.987727 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c0098-412f-4e99-a4af-ab0cdbd2d041 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.029000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a1390-3674-4104-bf1a-08db91135986 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.030939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23189f8-f2bf-4ef2-aff4-a9a1194e6f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.032036 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23189f8-f2bf-4ef2-aff4-a9a1194e6f41 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.032512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23189f8-f2bf-4ef2-aff4-a9a1194e6f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.032987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23189f8-f2bf-4ef2-aff4-a9a1194e6f41 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.035199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23189f8-f2bf-4ef2-aff4-a9a1194e6f41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.037931 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23189f8-f2bf-4ef2-aff4-a9a1194e6f41 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.040990 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 42/80] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:45.042430 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23189f8-f2bf-4ef2-aff4-a9a1194e6f41 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.054265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:45.067240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0402f-0be8-40a1-ac26-60362fa65877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.067890 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0402f-0be8-40a1-ac26-60362fa65877 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.068200 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0402f-0be8-40a1-ac26-60362fa65877 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.068473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0402f-0be8-40a1-ac26-60362fa65877 None admin] PATH_INFO: `/v3/users/7ded74de77c442b1853b2349fb3567ad` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.069644 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0402f-0be8-40a1-ac26-60362fa65877 None admin] RBAC: Authorizing `identity:get_user(user_id=7ded74de77c442b1853b2349fb3567ad)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.071178 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0402f-0be8-40a1-ac26-60362fa65877 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.072845 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 39/81] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:44 2022] GET /identity/v3/users/7ded74de77c442b1853b2349fb3567ad => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:45.073641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0402f-0be8-40a1-ac26-60362fa65877 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.132392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:45.187266 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23189f8-f2bf-4ef2-aff4-a9a1194e6f41 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.194803 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0402f-0be8-40a1-ac26-60362fa65877 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.228607 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069cb8e-247d-4a65-8305-a73a7ddf387a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.229388 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069cb8e-247d-4a65-8305-a73a7ddf387a None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.229746 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069cb8e-247d-4a65-8305-a73a7ddf387a None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.230021 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069cb8e-247d-4a65-8305-a73a7ddf387a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.234485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4c0fc-ffee-4359-81f2-51a92ee832cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.235857 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4c0fc-ffee-4359-81f2-51a92ee832cf None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.237262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4c0fc-ffee-4359-81f2-51a92ee832cf None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.237761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4c0fc-ffee-4359-81f2-51a92ee832cf None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.239018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4c0fc-ffee-4359-81f2-51a92ee832cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:45.239567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4c0fc-ffee-4359-81f2-51a92ee832cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.242249 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4c0fc-ffee-4359-81f2-51a92ee832cf None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.255035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 40/82] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:45.255941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4c0fc-ffee-4359-81f2-51a92ee832cf None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.308237 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069cb8e-247d-4a65-8305-a73a7ddf387a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.309745 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069cb8e-247d-4a65-8305-a73a7ddf387a None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.343441 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:45.384369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:45.418031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4c0fc-ffee-4359-81f2-51a92ee832cf None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.433777 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 43/83] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:45.434645 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069cb8e-247d-4a65-8305-a73a7ddf387a None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.460291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06d071-1a4c-4e0c-8b20-6b6e9296d2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.460963 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06d071-1a4c-4e0c-8b20-6b6e9296d2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.461239 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06d071-1a4c-4e0c-8b20-6b6e9296d2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.461514 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06d071-1a4c-4e0c-8b20-6b6e9296d2f0 None admin] PATH_INFO: `/v3/users/7ded74de77c442b1853b2349fb3567ad` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.462723 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06d071-1a4c-4e0c-8b20-6b6e9296d2f0 None admin] RBAC: Authorizing `identity:get_user(user_id=7ded74de77c442b1853b2349fb3567ad)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.464249 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06d071-1a4c-4e0c-8b20-6b6e9296d2f0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.465923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 41/84] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/users/7ded74de77c442b1853b2349fb3567ad => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:45.467642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06d071-1a4c-4e0c-8b20-6b6e9296d2f0 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.581902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069cb8e-247d-4a65-8305-a73a7ddf387a None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.622242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955d501-159f-495e-ba04-aa292e139e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.622811 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06d071-1a4c-4e0c-8b20-6b6e9296d2f0 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.622940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955d501-159f-495e-ba04-aa292e139e6b None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.623156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955d501-159f-495e-ba04-aa292e139e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.623432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955d501-159f-495e-ba04-aa292e139e6b None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.624166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955d501-159f-495e-ba04-aa292e139e6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.625856 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955d501-159f-495e-ba04-aa292e139e6b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.653069 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 44/85] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:29:45 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:45.654500 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955d501-159f-495e-ba04-aa292e139e6b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.658237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:45.681613 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:45.692721 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d122a43-311f-4656-8281-491db53908ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.693793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d122a43-311f-4656-8281-491db53908ff None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.694286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d122a43-311f-4656-8281-491db53908ff None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.694781 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d122a43-311f-4656-8281-491db53908ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.804446 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d122a43-311f-4656-8281-491db53908ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.806426 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d122a43-311f-4656-8281-491db53908ff None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.834896 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955d501-159f-495e-ba04-aa292e139e6b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.883781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54aa4b-5dc7-4455-94b8-dcc3c6eb1df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.884916 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54aa4b-5dc7-4455-94b8-dcc3c6eb1df1 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.885412 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54aa4b-5dc7-4455-94b8-dcc3c6eb1df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.885878 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54aa4b-5dc7-4455-94b8-dcc3c6eb1df1 None admin] PATH_INFO: `/v3/users/5f1827dde46f4915bfe3277a281b43bd` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.888468 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54aa4b-5dc7-4455-94b8-dcc3c6eb1df1 None admin] RBAC: Authorizing `identity:get_user(user_id=5f1827dde46f4915bfe3277a281b43bd)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.890158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54aa4b-5dc7-4455-94b8-dcc3c6eb1df1 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.893858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 45/86] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/users/5f1827dde46f4915bfe3277a281b43bd => generated 247 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:45.894742 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54aa4b-5dc7-4455-94b8-dcc3c6eb1df1 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.999431 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 42/87] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:46.002185 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d122a43-311f-4656-8281-491db53908ff None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:46.052963 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:46.054318 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:46.097850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54aa4b-5dc7-4455-94b8-dcc3c6eb1df1 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:46.168704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c36a53-c57a-4889-9c67-c6cb69e21c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:46.170121 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c36a53-c57a-4889-9c67-c6cb69e21c5d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:46.170851 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c36a53-c57a-4889-9c67-c6cb69e21c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:46.171584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c36a53-c57a-4889-9c67-c6cb69e21c5d None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:46.173979 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c36a53-c57a-4889-9c67-c6cb69e21c5d None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:46.177111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c36a53-c57a-4889-9c67-c6cb69e21c5d None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:46.180797 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 46/88] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:46.190143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c36a53-c57a-4889-9c67-c6cb69e21c5d None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:46.232385 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d122a43-311f-4656-8281-491db53908ff None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:46.297782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7aea4-2366-490e-b022-fb9f03bf2d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:46.299341 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7aea4-2366-490e-b022-fb9f03bf2d71 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:46.300173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7aea4-2366-490e-b022-fb9f03bf2d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:46.300904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7aea4-2366-490e-b022-fb9f03bf2d71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:46.357317 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c36a53-c57a-4889-9c67-c6cb69e21c5d None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:46.425042 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7aea4-2366-490e-b022-fb9f03bf2d71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:46.427859 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7aea4-2366-490e-b022-fb9f03bf2d71 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:46.431873 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7524b-6e0a-459d-a71b-50788793d083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:46.432928 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7524b-6e0a-459d-a71b-50788793d083 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:46.434616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7524b-6e0a-459d-a71b-50788793d083 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:46.435280 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7524b-6e0a-459d-a71b-50788793d083 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:46.464861 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:46.473836 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:46.551868 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7524b-6e0a-459d-a71b-50788793d083 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:46.554513 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7524b-6e0a-459d-a71b-50788793d083 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:46.640052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 43/89] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:46.641642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7aea4-2366-490e-b022-fb9f03bf2d71 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:46.763991 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 47/90] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:46.766223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7524b-6e0a-459d-a71b-50788793d083 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:46.875758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7aea4-2366-490e-b022-fb9f03bf2d71 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:46.897756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:46.935010 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:46.971293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351df9ee-4d13-4c9a-b828-ca59f5d96897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:46.972478 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351df9ee-4d13-4c9a-b828-ca59f5d96897 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:46.972951 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351df9ee-4d13-4c9a-b828-ca59f5d96897 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:46.973419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351df9ee-4d13-4c9a-b828-ca59f5d96897 None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:46.975563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351df9ee-4d13-4c9a-b828-ca59f5d96897 None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:46.978216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351df9ee-4d13-4c9a-b828-ca59f5d96897 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:46.982718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 44/91] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:46 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:46.984351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351df9ee-4d13-4c9a-b828-ca59f5d96897 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:47.012029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7524b-6e0a-459d-a71b-50788793d083 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:47.099331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732711a-a26d-47ff-9308-189aed42b81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:47.100487 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732711a-a26d-47ff-9308-189aed42b81a None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:47.100933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732711a-a26d-47ff-9308-189aed42b81a None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:47.101255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732711a-a26d-47ff-9308-189aed42b81a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:47.103202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e732711a-a26d-47ff-9308-189aed42b81a None admin] RBAC: Adding query filter params (role.id=4eebd8098ab44bfc84cd3de29a679bdf, scope.project.id=e02f7e20d2404b3888cc2e57180a901b, user.id=7ded74de77c442b1853b2349fb3567ad) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:47.103523 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e732711a-a26d-47ff-9308-189aed42b81a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:47.105071 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e732711a-a26d-47ff-9308-189aed42b81a None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:47.115611 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 48/92] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:46 2022] GET /identity/v3/role_assignments?user.id=7ded74de77c442b1853b2349fb3567ad&scope.project.id=e02f7e20d2404b3888cc2e57180a901b&role.id=4eebd8098ab44bfc84cd3de29a679bdf => generated 259 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:47.116552 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732711a-a26d-47ff-9308-189aed42b81a None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:47.143762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351df9ee-4d13-4c9a-b828-ca59f5d96897 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:47.211103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388ba48-bcea-4aa7-8731-8c1b03a4f21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:47.212863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388ba48-bcea-4aa7-8731-8c1b03a4f21c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:47.213689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388ba48-bcea-4aa7-8731-8c1b03a4f21c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:47.214458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388ba48-bcea-4aa7-8731-8c1b03a4f21c None admin] PATH_INFO: `/v3/projects/2ec1a8ad7bf7456ea8ef6d0d90bf47a7` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:47.216985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388ba48-bcea-4aa7-8731-8c1b03a4f21c None admin] RBAC: Authorizing `identity:get_project(project_id=2ec1a8ad7bf7456ea8ef6d0d90bf47a7)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:47.220355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388ba48-bcea-4aa7-8731-8c1b03a4f21c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:47.224465 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 45/93] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:46 2022] GET /identity/v3/projects/2ec1a8ad7bf7456ea8ef6d0d90bf47a7 => generated 312 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:47.226384 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388ba48-bcea-4aa7-8731-8c1b03a4f21c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:47.269690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732711a-a26d-47ff-9308-189aed42b81a None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:47.291293 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:47.332449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944854d2-cc73-4e26-b33d-e1672d0a7256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:47.333987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944854d2-cc73-4e26-b33d-e1672d0a7256 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:47.336269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944854d2-cc73-4e26-b33d-e1672d0a7256 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:47.336995 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944854d2-cc73-4e26-b33d-e1672d0a7256 None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:47.340968 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944854d2-cc73-4e26-b33d-e1672d0a7256 None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:47.345553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944854d2-cc73-4e26-b33d-e1672d0a7256 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:47.353761 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 49/94] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:47 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:47.355627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944854d2-cc73-4e26-b33d-e1672d0a7256 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:47.408245 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:47.503405 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388ba48-bcea-4aa7-8731-8c1b03a4f21c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:47.518705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944854d2-cc73-4e26-b33d-e1672d0a7256 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:47.567818 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce090b6-cb86-460b-af1a-5e2f4aa30020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:47.568507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce090b6-cb86-460b-af1a-5e2f4aa30020 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:47.568850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce090b6-cb86-460b-af1a-5e2f4aa30020 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:47.569135 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce090b6-cb86-460b-af1a-5e2f4aa30020 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:47.569971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce090b6-cb86-460b-af1a-5e2f4aa30020 None admin] RBAC: Adding query filter params (role.id=4eebd8098ab44bfc84cd3de29a679bdf, scope.project.id=2ec1a8ad7bf7456ea8ef6d0d90bf47a7, user.id=7ded74de77c442b1853b2349fb3567ad) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:47.570307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce090b6-cb86-460b-af1a-5e2f4aa30020 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:47.571810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce090b6-cb86-460b-af1a-5e2f4aa30020 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:47.574914 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5cdcd-2271-4ef1-a759-8bd9dd878d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:47.585225 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5cdcd-2271-4ef1-a759-8bd9dd878d57 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:47.586176 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5cdcd-2271-4ef1-a759-8bd9dd878d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:47.586910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5cdcd-2271-4ef1-a759-8bd9dd878d57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:47.588644 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d5cdcd-2271-4ef1-a759-8bd9dd878d57 None admin] RBAC: Adding query filter params (role.id=aedf4e1e5c7848099d7bd183778f5896, scope.project.id=e02f7e20d2404b3888cc2e57180a901b, user.id=7ded74de77c442b1853b2349fb3567ad) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:47.589298 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 50/95] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:47 2022] GET /identity/v3/role_assignments?user.id=7ded74de77c442b1853b2349fb3567ad&scope.project.id=2ec1a8ad7bf7456ea8ef6d0d90bf47a7&role.id=4eebd8098ab44bfc84cd3de29a679bdf => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:47.590133 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d5cdcd-2271-4ef1-a759-8bd9dd878d57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:47.590433 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce090b6-cb86-460b-af1a-5e2f4aa30020 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:47.593638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d5cdcd-2271-4ef1-a759-8bd9dd878d57 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:47.605385 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 46/96] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:47 2022] GET /identity/v3/role_assignments?user.id=7ded74de77c442b1853b2349fb3567ad&scope.project.id=e02f7e20d2404b3888cc2e57180a901b&role.id=aedf4e1e5c7848099d7bd183778f5896 => generated 259 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:47.715076 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:47.840371 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce090b6-cb86-460b-af1a-5e2f4aa30020 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:47.912043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73060f97-cdd2-4fc5-ae98-9d7194586658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:47.913063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73060f97-cdd2-4fc5-ae98-9d7194586658 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:47.913479 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73060f97-cdd2-4fc5-ae98-9d7194586658 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:47.913896 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73060f97-cdd2-4fc5-ae98-9d7194586658 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:47.915058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73060f97-cdd2-4fc5-ae98-9d7194586658 None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=e02f7e20d2404b3888cc2e57180a901b, user.id=5f1827dde46f4915bfe3277a281b43bd) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:47.915525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73060f97-cdd2-4fc5-ae98-9d7194586658 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:47.918247 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73060f97-cdd2-4fc5-ae98-9d7194586658 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:47.929423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 51/97] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:47 2022] GET /identity/v3/role_assignments?user.id=5f1827dde46f4915bfe3277a281b43bd&scope.project.id=e02f7e20d2404b3888cc2e57180a901b&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 259 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:49.019434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7414e14-7b0c-4ce3-87c0-8b963f1c37b4 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:49.020228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7414e14-7b0c-4ce3-87c0-8b963f1c37b4 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:49.020228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7414e14-7b0c-4ce3-87c0-8b963f1c37b4 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:49.021711 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 47/98] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:49.031571 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3c807-69ec-495f-99ac-15c005a2ea5c None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:49.032376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3c807-69ec-495f-99ac-15c005a2ea5c None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:49.032696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3c807-69ec-495f-99ac-15c005a2ea5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:49.064709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-92a3c807-69ec-495f-99ac-15c005a2ea5c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:49.066750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-92a3c807-69ec-495f-99ac-15c005a2ea5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:49.201108 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 52/99] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:49.661465 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7414e14-7b0c-4ce3-87c0-8b963f1c37b4 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:49.681970 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af096cf6-72ba-4c8e-a617-486c98985206 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:49.682285 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af096cf6-72ba-4c8e-a617-486c98985206 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:49.682724 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af096cf6-72ba-4c8e-a617-486c98985206 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:49.684713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 53/100] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:49.696219 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6708236-f4e2-4335-aa6a-86cb8ee71284 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:49.696542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6708236-f4e2-4335-aa6a-86cb8ee71284 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:49.696776 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6708236-f4e2-4335-aa6a-86cb8ee71284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:49.725888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-b6708236-f4e2-4335-aa6a-86cb8ee71284 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:49.727244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b6708236-f4e2-4335-aa6a-86cb8ee71284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:49.804444 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:49.877605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 54/101] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 30 14:29:49.917944 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7414e14-7b0c-4ce3-87c0-8b963f1c37b4 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:49.952452 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:49.974922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca1717-17ab-4f90-99e6-67fada9a2ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:49.975594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca1717-17ab-4f90-99e6-67fada9a2ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:49.975929 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca1717-17ab-4f90-99e6-67fada9a2ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:49.976234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca1717-17ab-4f90-99e6-67fada9a2ee4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:49.980811 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca1717-17ab-4f90-99e6-67fada9a2ee4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:49.982459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca1717-17ab-4f90-99e6-67fada9a2ee4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:49.984216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 48/102] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:49.990336 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6708236-f4e2-4335-aa6a-86cb8ee71284 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:50.151693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6708236-f4e2-4335-aa6a-86cb8ee71284 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:50.211493 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae68f7-bcf4-41bc-959e-864e1aa11d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:50.212082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae68f7-bcf4-41bc-959e-864e1aa11d50 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.212372 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae68f7-bcf4-41bc-959e-864e1aa11d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.212652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae68f7-bcf4-41bc-959e-864e1aa11d50 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.213343 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae68f7-bcf4-41bc-959e-864e1aa11d50 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:50.214989 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae68f7-bcf4-41bc-959e-864e1aa11d50 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:50.267331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 55/103] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Aug 30 14:29:49 2022] POST /identity/v3/projects => generated 302 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:50.375772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca1717-17ab-4f90-99e6-67fada9a2ee4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:50.536198 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca1717-17ab-4f90-99e6-67fada9a2ee4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:50.597615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15adb0-0098-434d-b0ed-ce3163e10649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:50.598723 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15adb0-0098-434d-b0ed-ce3163e10649 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.599232 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15adb0-0098-434d-b0ed-ce3163e10649 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.599702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15adb0-0098-434d-b0ed-ce3163e10649 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.616577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15adb0-0098-434d-b0ed-ce3163e10649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:50.622641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-da15adb0-0098-434d-b0ed-ce3163e10649 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:50.622641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:50.622641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:50.622641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:50.622641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:50.622641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:50.622641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:50.622641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:50.622641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:50.622641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:50.622641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:50.622641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:50.622641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:50.622641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:50.622641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:50.622641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:50.623598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:50.623598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:50.623598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:50.623598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:50.623598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:50.623598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:50.623598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:50.623598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:50.623598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:50.623598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:50.623598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:50.623598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:50.623598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:50.623598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:50.623598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:50.623598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:50.624400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:50.624400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:50.624400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:50.624400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:50.624400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:50.624400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:50.624400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:50.624400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:50.624400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:50.624400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:50.624400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:50.624400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:29:50.627760 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 49/104] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:50.634778 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae68f7-bcf4-41bc-959e-864e1aa11d50 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:50.655376 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028afb53-bdac-4604-ac1e-63b6d9c5decb None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.656065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028afb53-bdac-4604-ac1e-63b6d9c5decb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.656576 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028afb53-bdac-4604-ac1e-63b6d9c5decb None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.659326 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 50/105] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:50.665243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213bae4-da96-483a-be0a-1674ee0b72d9 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.665933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213bae4-da96-483a-be0a-1674ee0b72d9 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.666403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213bae4-da96-483a-be0a-1674ee0b72d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.694529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:50.709577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8213bae4-da96-483a-be0a-1674ee0b72d9 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:50.711135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8213bae4-da96-483a-be0a-1674ee0b72d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:50.788573 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae68f7-bcf4-41bc-959e-864e1aa11d50 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:50.824438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:50.834105 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2e603-ca76-437f-b01e-f9c600982fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:50.835294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2e603-ca76-437f-b01e-f9c600982fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.835648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2e603-ca76-437f-b01e-f9c600982fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.836035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2e603-ca76-437f-b01e-f9c600982fe7 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.836816 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2e603-ca76-437f-b01e-f9c600982fe7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:50.837195 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2e603-ca76-437f-b01e-f9c600982fe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:50.838720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2e603-ca76-437f-b01e-f9c600982fe7 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:50.849370 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 56/106] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:50.850560 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834270b-ca46-4d7f-a0aa-5a1819694b5b None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.850924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834270b-ca46-4d7f-a0aa-5a1819694b5b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.851290 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834270b-ca46-4d7f-a0aa-5a1819694b5b None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.857120 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 57/107] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:50.859693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834270b-ca46-4d7f-a0aa-5a1819694b5b None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:50.871888 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 51/108] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:50.874095 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4a8e5-6a8c-4d93-8321-7aee21407bf5 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.874573 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4a8e5-6a8c-4d93-8321-7aee21407bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.875561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4a8e5-6a8c-4d93-8321-7aee21407bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.914729 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-6ff4a8e5-6a8c-4d93-8321-7aee21407bf5 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:50.915959 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6ff4a8e5-6a8c-4d93-8321-7aee21407bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:51.076956 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:51.099661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 52/109] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:51.112636 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834270b-ca46-4d7f-a0aa-5a1819694b5b None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:51.168941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ad328-2263-4982-a4ca-c8a17043a26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:51.169742 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ad328-2263-4982-a4ca-c8a17043a26c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.169997 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ad328-2263-4982-a4ca-c8a17043a26c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.170277 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ad328-2263-4982-a4ca-c8a17043a26c None admin] PATH_INFO: `/v3/users/b9690d46a2454005a20d429acb3d3525` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.197555 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ad328-2263-4982-a4ca-c8a17043a26c None admin] RBAC: Authorizing `identity:get_user(user_id=b9690d46a2454005a20d429acb3d3525)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:51.199194 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ad328-2263-4982-a4ca-c8a17043a26c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:51.201253 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 58/110] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:50 2022] GET /identity/v3/users/b9690d46a2454005a20d429acb3d3525 => generated 289 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:51.208768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4a8e5-6a8c-4d93-8321-7aee21407bf5 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.261715 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ad328-2263-4982-a4ca-c8a17043a26c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.375547 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ad328-2263-4982-a4ca-c8a17043a26c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:51.400986 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4a8e5-6a8c-4d93-8321-7aee21407bf5 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:51.417047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1de9c-bfdc-4a20-96ef-0fcc736210bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:51.418087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1de9c-bfdc-4a20-96ef-0fcc736210bc None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.418575 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1de9c-bfdc-4a20-96ef-0fcc736210bc None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.419065 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1de9c-bfdc-4a20-96ef-0fcc736210bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.422554 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:51.434894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1de9c-bfdc-4a20-96ef-0fcc736210bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:51.441414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-63c1de9c-bfdc-4a20-96ef-0fcc736210bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:51.441414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:51.441414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:51.441414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:51.441414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:51.441414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:51.441414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:51.441414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:51.441414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:51.441414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:51.441414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:51.441414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:51.441414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:51.441414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:51.441414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:51.441414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:51.442850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:51.442850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:51.442850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:51.442850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:51.442850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:51.442850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:51.442850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:51.442850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:51.442850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:51.442850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:51.442850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:51.442850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:51.442850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:51.442850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:51.442850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:51.442850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:51.443639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:51.443639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:51.443639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:51.443639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:51.443639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:51.443639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:51.443639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:51.443639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:51.443639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:51.443639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:51.443639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:51.443639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:29:51.446360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 59/111] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:29:51.450489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636c90d-97c3-4a70-8e59-380a3e1a42c8 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.450796 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636c90d-97c3-4a70-8e59-380a3e1a42c8 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.451024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636c90d-97c3-4a70-8e59-380a3e1a42c8 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.456644 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 60/112] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:51.458184 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636c90d-97c3-4a70-8e59-380a3e1a42c8 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.462906 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07b6be-d6d4-4fb9-bfae-b3819b10c836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:51.464308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07b6be-d6d4-4fb9-bfae-b3819b10c836 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.464308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07b6be-d6d4-4fb9-bfae-b3819b10c836 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.464308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07b6be-d6d4-4fb9-bfae-b3819b10c836 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.464550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:51.544353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07b6be-d6d4-4fb9-bfae-b3819b10c836 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:51.546317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07b6be-d6d4-4fb9-bfae-b3819b10c836 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:51.615187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636c90d-97c3-4a70-8e59-380a3e1a42c8 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:51.657527 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07442ef6-0deb-452e-8df7-5e6523fa6982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:51.658300 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07442ef6-0deb-452e-8df7-5e6523fa6982 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.658673 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07442ef6-0deb-452e-8df7-5e6523fa6982 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.659066 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07442ef6-0deb-452e-8df7-5e6523fa6982 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.671958 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07442ef6-0deb-452e-8df7-5e6523fa6982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:51.678940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-07442ef6-0deb-452e-8df7-5e6523fa6982 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:51.678940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:51.678940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:51.678940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:51.678940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:51.678940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:51.678940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:51.678940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:51.678940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:51.678940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:51.678940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:51.678940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:51.678940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:51.678940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:51.678940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:51.678940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:51.680023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:51.680023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:51.680023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:51.680023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:51.680023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:51.680023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:51.680023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:51.680023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:51.680023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:51.680023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:51.680023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:51.680023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:51.680023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:51.680023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:51.680023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:51.680023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:51.680828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:51.680828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:51.680828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:51.680828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:51.680828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:51.680828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:51.680828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:51.680828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:51.680828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:51.680828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:51.680828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:51.680828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:29:51.680828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 61/113] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:51.682000 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07442ef6-0deb-452e-8df7-5e6523fa6982 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.683149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 53/114] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:51.684773 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33715463-fb3d-4d5d-b427-685cb9ec245b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.685388 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33715463-fb3d-4d5d-b427-685cb9ec245b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.685880 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33715463-fb3d-4d5d-b427-685cb9ec245b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.711471 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-33715463-fb3d-4d5d-b427-685cb9ec245b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:51.713539 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-33715463-fb3d-4d5d-b427-685cb9ec245b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:51.716532 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:51.765658 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:51.802121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 54/115] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:51.804439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33715463-fb3d-4d5d-b427-685cb9ec245b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.812836 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07442ef6-0deb-452e-8df7-5e6523fa6982 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:51.853847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a753e1-018e-4118-beff-3bfff0e8c90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:51.854809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a753e1-018e-4118-beff-3bfff0e8c90d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.855334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a753e1-018e-4118-beff-3bfff0e8c90d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.855896 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a753e1-018e-4118-beff-3bfff0e8c90d None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.856872 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a753e1-018e-4118-beff-3bfff0e8c90d None admin] RBAC: Adding query filter params (name=member) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:51.857394 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a753e1-018e-4118-beff-3bfff0e8c90d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:51.859054 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a753e1-018e-4118-beff-3bfff0e8c90d None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:51.869071 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 62/116] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:51.875349 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a753e1-018e-4118-beff-3bfff0e8c90d None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.923879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33715463-fb3d-4d5d-b427-685cb9ec245b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:51.963594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709704c4-2b13-4b0c-a68b-bad62bb32860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:51.964273 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709704c4-2b13-4b0c-a68b-bad62bb32860 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.964553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709704c4-2b13-4b0c-a68b-bad62bb32860 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.964819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709704c4-2b13-4b0c-a68b-bad62bb32860 None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.966086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709704c4-2b13-4b0c-a68b-bad62bb32860 None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:51.967581 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709704c4-2b13-4b0c-a68b-bad62bb32860 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:51.970031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 55/117] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:51.971338 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224854fe-706e-4772-911c-91b79d9f5779 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.971618 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224854fe-706e-4772-911c-91b79d9f5779 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.971891 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224854fe-706e-4772-911c-91b79d9f5779 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.973541 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 56/118] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:51.974312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224854fe-706e-4772-911c-91b79d9f5779 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.984420 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:52.000576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a753e1-018e-4118-beff-3bfff0e8c90d None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.040866 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:52.057467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65959562-a1be-4000-886b-0ec07f33c6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.058411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65959562-a1be-4000-886b-0ec07f33c6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.058700 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65959562-a1be-4000-886b-0ec07f33c6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.058981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65959562-a1be-4000-886b-0ec07f33c6f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.060255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65959562-a1be-4000-886b-0ec07f33c6f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:52.060588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65959562-a1be-4000-886b-0ec07f33c6f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.062375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65959562-a1be-4000-886b-0ec07f33c6f7 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.074575 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 63/119] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:52.075466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65959562-a1be-4000-886b-0ec07f33c6f7 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.106244 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224854fe-706e-4772-911c-91b79d9f5779 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.145482 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e996eaa-206f-486b-b88b-5bbb6b2cbfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.146159 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e996eaa-206f-486b-b88b-5bbb6b2cbfff None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.146444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e996eaa-206f-486b-b88b-5bbb6b2cbfff None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.147067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e996eaa-206f-486b-b88b-5bbb6b2cbfff None admin] PATH_INFO: `/v3/users/7ded74de77c442b1853b2349fb3567ad` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.148288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e996eaa-206f-486b-b88b-5bbb6b2cbfff None admin] RBAC: Authorizing `identity:get_user(user_id=7ded74de77c442b1853b2349fb3567ad)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.149766 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e996eaa-206f-486b-b88b-5bbb6b2cbfff None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.151405 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 57/120] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/users/7ded74de77c442b1853b2349fb3567ad => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:52.157308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa35a6-4b90-4212-9560-3a674b4a7ae8 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.157940 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa35a6-4b90-4212-9560-3a674b4a7ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.157940 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa35a6-4b90-4212-9560-3a674b4a7ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.176666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-cbfa35a6-4b90-4212-9560-3a674b4a7ae8 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:52.177824 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cbfa35a6-4b90-4212-9560-3a674b4a7ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:52.201484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65959562-a1be-4000-886b-0ec07f33c6f7 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.246119 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c90816-95b7-4e96-95b7-0ef1489ff075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.246755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c90816-95b7-4e96-95b7-0ef1489ff075 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.247070 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c90816-95b7-4e96-95b7-0ef1489ff075 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.247378 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c90816-95b7-4e96-95b7-0ef1489ff075 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.248059 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c90816-95b7-4e96-95b7-0ef1489ff075 None admin] RBAC: Adding query filter params (role.id=89f1b9db96764371860f845c00419e7a, scope.project.id=e02f7e20d2404b3888cc2e57180a901b, user.id=b9690d46a2454005a20d429acb3d3525) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:52.248369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c90816-95b7-4e96-95b7-0ef1489ff075 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.249841 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c90816-95b7-4e96-95b7-0ef1489ff075 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.250772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 58/121] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:52.252513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa35a6-4b90-4212-9560-3a674b4a7ae8 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.264503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 64/122] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/role_assignments?user.id=b9690d46a2454005a20d429acb3d3525&scope.project.id=e02f7e20d2404b3888cc2e57180a901b&role.id=89f1b9db96764371860f845c00419e7a => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:52.265566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c90816-95b7-4e96-95b7-0ef1489ff075 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.276421 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:52.384726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa35a6-4b90-4212-9560-3a674b4a7ae8 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.430594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:52.432264 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9da432-7339-450a-9290-c34610f10e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.433709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9da432-7339-450a-9290-c34610f10e00 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.434732 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9da432-7339-450a-9290-c34610f10e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.435155 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9da432-7339-450a-9290-c34610f10e00 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.449589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9da432-7339-450a-9290-c34610f10e00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:52.455623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-eb9da432-7339-450a-9290-c34610f10e00 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:52.455623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:52.455623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:52.455623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:52.455623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:52.455623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:52.455623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:52.455623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:52.455623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:52.455623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:52.455623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:52.455623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:52.455623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:52.455623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:52.455623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:52.455623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:52.456565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:52.456565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:52.456565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:52.456565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:52.456565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:52.456565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:52.456565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:52.456565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:52.456565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:52.456565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:52.456565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:52.456565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:52.456565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:52.456565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:52.456565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:52.456565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:52.457320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:52.457320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:52.457320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:52.457320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:52.457320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:52.457320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:52.457320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:52.457320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:52.457320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:52.457320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:52.457320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:52.457320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:29:52.457320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 59/123] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:52.457974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9da432-7339-450a-9290-c34610f10e00 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.522907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c90816-95b7-4e96-95b7-0ef1489ff075 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.552942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:52.568145 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4f6a3-7080-4b6d-9df9-63bfc30552f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.569809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4f6a3-7080-4b6d-9df9-63bfc30552f0 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.570302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4f6a3-7080-4b6d-9df9-63bfc30552f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.570781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4f6a3-7080-4b6d-9df9-63bfc30552f0 None admin] PATH_INFO: `/v3/users/7ded74de77c442b1853b2349fb3567ad` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.572656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4f6a3-7080-4b6d-9df9-63bfc30552f0 None admin] RBAC: Authorizing `identity:get_user(user_id=7ded74de77c442b1853b2349fb3567ad)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.574910 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4f6a3-7080-4b6d-9df9-63bfc30552f0 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.577477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 65/124] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/users/7ded74de77c442b1853b2349fb3567ad => generated 275 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:52.578514 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4f6a3-7080-4b6d-9df9-63bfc30552f0 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.603805 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9da432-7339-450a-9290-c34610f10e00 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.645403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcf919-41a0-472d-bf19-13d0b1e4a35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.646167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcf919-41a0-472d-bf19-13d0b1e4a35b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.646537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcf919-41a0-472d-bf19-13d0b1e4a35b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.647107 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcf919-41a0-472d-bf19-13d0b1e4a35b None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.648386 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcf919-41a0-472d-bf19-13d0b1e4a35b None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.650029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcf919-41a0-472d-bf19-13d0b1e4a35b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 60/125] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:52.653020 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcf919-41a0-472d-bf19-13d0b1e4a35b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.715340 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4f6a3-7080-4b6d-9df9-63bfc30552f0 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.737853 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:52.759599 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefd61d-68dc-4426-a971-63aa60b18b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.760414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefd61d-68dc-4426-a971-63aa60b18b92 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.760745 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefd61d-68dc-4426-a971-63aa60b18b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.761075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefd61d-68dc-4426-a971-63aa60b18b92 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.761851 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daefd61d-68dc-4426-a971-63aa60b18b92 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:52.762235 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daefd61d-68dc-4426-a971-63aa60b18b92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.763862 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daefd61d-68dc-4426-a971-63aa60b18b92 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.774769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 66/126] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:52.775847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefd61d-68dc-4426-a971-63aa60b18b92 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.788848 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcf919-41a0-472d-bf19-13d0b1e4a35b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.831292 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:52.840743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b753c-d80a-4f53-9397-43581db0582f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.841486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b753c-d80a-4f53-9397-43581db0582f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.841871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b753c-d80a-4f53-9397-43581db0582f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.842240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b753c-d80a-4f53-9397-43581db0582f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.859473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b753c-d80a-4f53-9397-43581db0582f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:52.867079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-426b753c-d80a-4f53-9397-43581db0582f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:52.867079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:52.867079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:52.867079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:52.867079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:52.867079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:52.867079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:52.867079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:52.867079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:52.867079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:52.867079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:52.867079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:52.867079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:52.867079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:52.867079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:52.867079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:52.868087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:52.868087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:52.868087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:52.868087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:52.868087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:52.868087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:52.868087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:52.868087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:52.868087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:52.868087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:52.868087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:52.868087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:52.868087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:52.868087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:52.868087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:52.868087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:52.869141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:52.869141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:52.869141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:52.869141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:52.869141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:52.869141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:52.869141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:52.869141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:52.869141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:52.869141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:52.869141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:52.869141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:29:52.869141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 61/127] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:52.869957 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b753c-d80a-4f53-9397-43581db0582f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.908544 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefd61d-68dc-4426-a971-63aa60b18b92 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.964151 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88182792-f58a-4362-8657-a12820403a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.965118 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88182792-f58a-4362-8657-a12820403a51 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.965839 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88182792-f58a-4362-8657-a12820403a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.966322 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88182792-f58a-4362-8657-a12820403a51 None admin] PATH_INFO: `/v3/projects/2ec1a8ad7bf7456ea8ef6d0d90bf47a7` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.968388 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88182792-f58a-4362-8657-a12820403a51 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec1a8ad7bf7456ea8ef6d0d90bf47a7)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.971047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88182792-f58a-4362-8657-a12820403a51 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.979390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 67/128] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/projects/2ec1a8ad7bf7456ea8ef6d0d90bf47a7 => generated 312 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:52.982456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88182792-f58a-4362-8657-a12820403a51 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:53.016247 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b753c-d80a-4f53-9397-43581db0582f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:53.044203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:53.053942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d7c29-552d-4e21-91b6-b22c6272760a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:53.054681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d7c29-552d-4e21-91b6-b22c6272760a None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.054907 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d7c29-552d-4e21-91b6-b22c6272760a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.055233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d7c29-552d-4e21-91b6-b22c6272760a None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b/users/7ded74de77c442b1853b2349fb3567ad/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.057318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d7c29-552d-4e21-91b6-b22c6272760a None admin] RBAC: Authorizing `identity:create_grant(project_id=e02f7e20d2404b3888cc2e57180a901b, user_id=7ded74de77c442b1853b2349fb3567ad, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:53.058769 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d7c29-552d-4e21-91b6-b22c6272760a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:53.074687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 62/129] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:29:52 2022] PUT /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b/users/7ded74de77c442b1853b2349fb3567ad/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:29:53.075643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d7c29-552d-4e21-91b6-b22c6272760a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:53.114291 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88182792-f58a-4362-8657-a12820403a51 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:53.158677 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2341615-77d8-4252-9ca2-e31371b36ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:53.159405 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2341615-77d8-4252-9ca2-e31371b36ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.159969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2341615-77d8-4252-9ca2-e31371b36ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.159969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2341615-77d8-4252-9ca2-e31371b36ee1 None admin] PATH_INFO: `/v3/users/7ded74de77c442b1853b2349fb3567ad` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.161325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2341615-77d8-4252-9ca2-e31371b36ee1 None admin] RBAC: Authorizing `identity:get_user(user_id=7ded74de77c442b1853b2349fb3567ad)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:53.162923 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2341615-77d8-4252-9ca2-e31371b36ee1 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:53.164263 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:53.164834 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 68/130] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/users/7ded74de77c442b1853b2349fb3567ad => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:53.165729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2341615-77d8-4252-9ca2-e31371b36ee1 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:53.237179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d7c29-552d-4e21-91b6-b22c6272760a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:53.280818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c2872-77e6-4a86-b0f3-1f1ec69bbb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:53.281773 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c2872-77e6-4a86-b0f3-1f1ec69bbb1f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.282118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c2872-77e6-4a86-b0f3-1f1ec69bbb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.282406 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c2872-77e6-4a86-b0f3-1f1ec69bbb1f None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.283624 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c2872-77e6-4a86-b0f3-1f1ec69bbb1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:53.283978 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c2872-77e6-4a86-b0f3-1f1ec69bbb1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:53.286070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c2872-77e6-4a86-b0f3-1f1ec69bbb1f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:53.302657 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 63/131] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:53.303325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2341615-77d8-4252-9ca2-e31371b36ee1 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:53.304029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715be0a0-96c0-4645-958c-48372ed857eb None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.304325 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715be0a0-96c0-4645-958c-48372ed857eb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.304605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715be0a0-96c0-4645-958c-48372ed857eb None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.305864 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 64/132] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:53.313480 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715be0a0-96c0-4645-958c-48372ed857eb None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:53.356434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:53.365044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0c48d-c3d0-44ab-b5e3-b8890bb9549e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:53.366748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0c48d-c3d0-44ab-b5e3-b8890bb9549e None admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.367219 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0c48d-c3d0-44ab-b5e3-b8890bb9549e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.367591 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0c48d-c3d0-44ab-b5e3-b8890bb9549e None admin] PATH_INFO: `/v3/projects/2ec1a8ad7bf7456ea8ef6d0d90bf47a7/users/7ded74de77c442b1853b2349fb3567ad/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.370042 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b0c48d-c3d0-44ab-b5e3-b8890bb9549e None admin] RBAC: Authorizing `identity:create_grant(project_id=2ec1a8ad7bf7456ea8ef6d0d90bf47a7, user_id=7ded74de77c442b1853b2349fb3567ad, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:53.371706 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b0c48d-c3d0-44ab-b5e3-b8890bb9549e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:53.391382 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 69/133] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:29:53 2022] PUT /identity/v3/projects/2ec1a8ad7bf7456ea8ef6d0d90bf47a7/users/7ded74de77c442b1853b2349fb3567ad/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:29:53.392661 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f7e25-3fe8-4fe6-b030-3156e21148fa None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.393076 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f7e25-3fe8-4fe6-b030-3156e21148fa None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.393463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f7e25-3fe8-4fe6-b030-3156e21148fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.416857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-f84f7e25-3fe8-4fe6-b030-3156e21148fa None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:53.418386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f84f7e25-3fe8-4fe6-b030-3156e21148fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:53.467167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715be0a0-96c0-4645-958c-48372ed857eb None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:53.483963 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:53.506132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 70/134] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:53.507287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f7e25-3fe8-4fe6-b030-3156e21148fa None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:53.540851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d79c4f-0865-4c92-8f6b-adfe1f4fb883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:53.541817 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d79c4f-0865-4c92-8f6b-adfe1f4fb883 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.542237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d79c4f-0865-4c92-8f6b-adfe1f4fb883 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.542650 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d79c4f-0865-4c92-8f6b-adfe1f4fb883 None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.548950 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d79c4f-0865-4c92-8f6b-adfe1f4fb883 None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:53.550772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d79c4f-0865-4c92-8f6b-adfe1f4fb883 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:53.552867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 65/135] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:53 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:53.559414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d79c4f-0865-4c92-8f6b-adfe1f4fb883 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:53.683603 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f7e25-3fe8-4fe6-b030-3156e21148fa None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:53.696936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:53.744991 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc20603-b097-4c94-ba35-366b2814ae7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:53.745706 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc20603-b097-4c94-ba35-366b2814ae7b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.746142 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc20603-b097-4c94-ba35-366b2814ae7b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.746624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc20603-b097-4c94-ba35-366b2814ae7b None admin] PATH_INFO: `/v3/users/5f1827dde46f4915bfe3277a281b43bd` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.748285 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d79c4f-0865-4c92-8f6b-adfe1f4fb883 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:53.749062 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc20603-b097-4c94-ba35-366b2814ae7b None admin] RBAC: Authorizing `identity:get_user(user_id=5f1827dde46f4915bfe3277a281b43bd)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:53.753415 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc20603-b097-4c94-ba35-366b2814ae7b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:53.756608 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 71/136] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:53 2022] GET /identity/v3/users/5f1827dde46f4915bfe3277a281b43bd => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:53.780247 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc20603-b097-4c94-ba35-366b2814ae7b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:53.798789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b680154-240a-4303-981d-121f58992be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:53.799769 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b680154-240a-4303-981d-121f58992be1 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.800216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b680154-240a-4303-981d-121f58992be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.800643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b680154-240a-4303-981d-121f58992be1 None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b/users/7ded74de77c442b1853b2349fb3567ad/roles/aedf4e1e5c7848099d7bd183778f5896` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.807926 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b680154-240a-4303-981d-121f58992be1 None admin] RBAC: Authorizing `identity:create_grant(project_id=e02f7e20d2404b3888cc2e57180a901b, user_id=7ded74de77c442b1853b2349fb3567ad, role_id=aedf4e1e5c7848099d7bd183778f5896)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:53.809647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b680154-240a-4303-981d-121f58992be1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:53.836498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 66/137] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:29:53 2022] PUT /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b/users/7ded74de77c442b1853b2349fb3567ad/roles/aedf4e1e5c7848099d7bd183778f5896 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:29:53.941984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc20603-b097-4c94-ba35-366b2814ae7b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:53.989180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c2ac4-4fa3-4f3f-bc9c-a9e6c157de05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:53.990160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c2ac4-4fa3-4f3f-bc9c-a9e6c157de05 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.990603 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c2ac4-4fa3-4f3f-bc9c-a9e6c157de05 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.991013 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c2ac4-4fa3-4f3f-bc9c-a9e6c157de05 None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.992513 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c2ac4-4fa3-4f3f-bc9c-a9e6c157de05 None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:53.994291 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c2ac4-4fa3-4f3f-bc9c-a9e6c157de05 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:53.996320 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 72/138] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:53 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:54.005877 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b680154-240a-4303-981d-121f58992be1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:54.070167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:54.127040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c2ac4-4fa3-4f3f-bc9c-a9e6c157de05 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:54.181001 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b680154-240a-4303-981d-121f58992be1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:54.232156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:54.246499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65aa71b-dc83-4670-bb61-e74a0bb8f085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:54.247208 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aa71b-dc83-4670-bb61-e74a0bb8f085 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:54.247494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aa71b-dc83-4670-bb61-e74a0bb8f085 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:54.247770 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aa71b-dc83-4670-bb61-e74a0bb8f085 None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b/users/5f1827dde46f4915bfe3277a281b43bd/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:54.254995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65aa71b-dc83-4670-bb61-e74a0bb8f085 None admin] RBAC: Authorizing `identity:create_grant(project_id=e02f7e20d2404b3888cc2e57180a901b, user_id=5f1827dde46f4915bfe3277a281b43bd, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:54.258561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65aa71b-dc83-4670-bb61-e74a0bb8f085 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:54.281556 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 67/139] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:29:54 2022] PUT /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b/users/5f1827dde46f4915bfe3277a281b43bd/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:29:54.371833 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c2ac4-4fa3-4f3f-bc9c-a9e6c157de05 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:54.416368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79e0d2-b988-4dfb-8f2b-c74a88194566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:54.417031 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79e0d2-b988-4dfb-8f2b-c74a88194566 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:54.417309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79e0d2-b988-4dfb-8f2b-c74a88194566 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:54.417583 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79e0d2-b988-4dfb-8f2b-c74a88194566 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:54.418815 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79e0d2-b988-4dfb-8f2b-c74a88194566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:54.420321 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79e0d2-b988-4dfb-8f2b-c74a88194566 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:54.422329 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 73/140] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:54.427875 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65aa71b-dc83-4670-bb61-e74a0bb8f085 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:54.564551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65aa71b-dc83-4670-bb61-e74a0bb8f085 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:54.566391 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:54.570566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5132d30-41db-483b-92b6-cda3d75d29b1 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:54.570858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5132d30-41db-483b-92b6-cda3d75d29b1 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:54.571118 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5132d30-41db-483b-92b6-cda3d75d29b1 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:54.572526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 74/141] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:54.578411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708fe17-375c-47c1-b59b-74e22e62e801 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:54.578729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708fe17-375c-47c1-b59b-74e22e62e801 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:54.579211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708fe17-375c-47c1-b59b-74e22e62e801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:54.605453 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-d708fe17-375c-47c1-b59b-74e22e62e801 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:54.607449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d708fe17-375c-47c1-b59b-74e22e62e801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:54.612717 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c23bb-0480-4f8a-a2e1-68e2907527dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:54.613459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c23bb-0480-4f8a-a2e1-68e2907527dc None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:54.613816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c23bb-0480-4f8a-a2e1-68e2907527dc None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:54.614161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c23bb-0480-4f8a-a2e1-68e2907527dc None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:54.614917 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c23bb-0480-4f8a-a2e1-68e2907527dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:54.616539 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c23bb-0480-4f8a-a2e1-68e2907527dc None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:54.668040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 68/142] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:29:54 2022] POST /identity/v3/users => generated 283 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:54.765561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 75/143] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:55.261323 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c23bb-0480-4f8a-a2e1-68e2907527dc None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:55.396784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c23bb-0480-4f8a-a2e1-68e2907527dc None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:55.439708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1300d-8ecd-4eaa-a881-a9212af8d599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:55.440829 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1300d-8ecd-4eaa-a881-a9212af8d599 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:55.441327 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1300d-8ecd-4eaa-a881-a9212af8d599 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:55.441827 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1300d-8ecd-4eaa-a881-a9212af8d599 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:55.457424 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c1300d-8ecd-4eaa-a881-a9212af8d599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:55.467067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-17c1300d-8ecd-4eaa-a881-a9212af8d599 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:55.467067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:55.467067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:55.467067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:55.467067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:55.467067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:55.467067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:55.467067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:55.467067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:55.467067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:55.467067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:55.467067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:55.467067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:55.467067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:55.467067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:55.467067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:55.468536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:55.468536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:55.468536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:55.468536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:55.468536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:55.468536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:55.468536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:55.468536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:55.468536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:55.468536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:55.468536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:55.468536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:55.468536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:55.468536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:55.468536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:55.468536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:55.469775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:55.469775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:55.469775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:55.469775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:55.469775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:55.469775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:55.469775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:55.469775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:55.469775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:55.469775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:55.469775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:55.469775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:29:55.475069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 69/144] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:55.485276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708fe17-375c-47c1-b59b-74e22e62e801 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:55.568466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:55.669007 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708fe17-375c-47c1-b59b-74e22e62e801 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:55.744423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3c85f-2176-4310-8a2c-02b35f4bda2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:55.745082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3c85f-2176-4310-8a2c-02b35f4bda2e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:55.745365 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3c85f-2176-4310-8a2c-02b35f4bda2e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:55.745638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3c85f-2176-4310-8a2c-02b35f4bda2e None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:55.746343 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3c85f-2176-4310-8a2c-02b35f4bda2e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:55.746648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3c85f-2176-4310-8a2c-02b35f4bda2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:55.748158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3c85f-2176-4310-8a2c-02b35f4bda2e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:55.760505 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 76/145] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:55.769211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1300d-8ecd-4eaa-a881-a9212af8d599 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:55.807352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:55.922383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1300d-8ecd-4eaa-a881-a9212af8d599 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:55.965248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf296ab-9541-4b83-a378-5969998d9f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:55.966204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf296ab-9541-4b83-a378-5969998d9f5a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:55.966623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf296ab-9541-4b83-a378-5969998d9f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:55.967052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf296ab-9541-4b83-a378-5969998d9f5a None admin] PATH_INFO: `/v3/users/b9690d46a2454005a20d429acb3d3525` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:55.968618 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf296ab-9541-4b83-a378-5969998d9f5a None admin] RBAC: Authorizing `identity:get_user(user_id=b9690d46a2454005a20d429acb3d3525)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:55.970485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf296ab-9541-4b83-a378-5969998d9f5a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:55.976702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 70/146] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:55 2022] GET /identity/v3/users/b9690d46a2454005a20d429acb3d3525 => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:55.982910 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3c85f-2176-4310-8a2c-02b35f4bda2e None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:56.072830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a038a7-0d47-4adf-8b94-a6477f51c844 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:56.073471 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a038a7-0d47-4adf-8b94-a6477f51c844 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:56.073961 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a038a7-0d47-4adf-8b94-a6477f51c844 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:56.075594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 71/147] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:56.083880 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d84e0c-a121-474d-976c-8f6380a54396 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:56.084884 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d84e0c-a121-474d-976c-8f6380a54396 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:56.085353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d84e0c-a121-474d-976c-8f6380a54396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:56.120621 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f8d84e0c-a121-474d-976c-8f6380a54396 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:56.122207 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f8d84e0c-a121-474d-976c-8f6380a54396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:56.196125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3c85f-2176-4310-8a2c-02b35f4bda2e None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:56.197777 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:56.233871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 72/148] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:56.255043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84802652-735f-4189-aae8-ff11103b4efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:56.256154 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802652-735f-4189-aae8-ff11103b4efc None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:56.256631 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802652-735f-4189-aae8-ff11103b4efc None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:56.257108 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802652-735f-4189-aae8-ff11103b4efc None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:56.258574 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84802652-735f-4189-aae8-ff11103b4efc None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:56.260359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84802652-735f-4189-aae8-ff11103b4efc None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:56.262668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 77/149] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:55 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:56.269190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d84e0c-a121-474d-976c-8f6380a54396 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:56.294376 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:56.424090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d84e0c-a121-474d-976c-8f6380a54396 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:56.468730 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deff02a-8e14-4aa9-8cff-fbb79435f29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:56.469350 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deff02a-8e14-4aa9-8cff-fbb79435f29e None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:56.469770 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deff02a-8e14-4aa9-8cff-fbb79435f29e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:56.470194 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deff02a-8e14-4aa9-8cff-fbb79435f29e None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b/users/b9690d46a2454005a20d429acb3d3525/roles/89f1b9db96764371860f845c00419e7a` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:56.473731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deff02a-8e14-4aa9-8cff-fbb79435f29e None admin] RBAC: Authorizing `identity:create_grant(project_id=e02f7e20d2404b3888cc2e57180a901b, user_id=b9690d46a2454005a20d429acb3d3525, role_id=89f1b9db96764371860f845c00419e7a)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:56.475261 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deff02a-8e14-4aa9-8cff-fbb79435f29e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:56.495573 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 73/150] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:29:56 2022] PUT /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b/users/b9690d46a2454005a20d429acb3d3525/roles/89f1b9db96764371860f845c00419e7a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:29:56.900830 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84802652-735f-4189-aae8-ff11103b4efc None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:57.050038 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84802652-735f-4189-aae8-ff11103b4efc None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:57.089753 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96b8a5-265a-4ee3-984c-21b1d298d5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:57.090818 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96b8a5-265a-4ee3-984c-21b1d298d5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.091294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96b8a5-265a-4ee3-984c-21b1d298d5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.091759 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96b8a5-265a-4ee3-984c-21b1d298d5f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.109763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a96b8a5-265a-4ee3-984c-21b1d298d5f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:57.116030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-8a96b8a5-265a-4ee3-984c-21b1d298d5f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:57.116030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:57.116030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:57.116030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:57.116030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:57.116030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:57.116030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:57.116030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:57.116030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:57.116030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:57.116030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:57.116030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:57.116030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:57.116030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:57.116030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:57.116030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:57.116945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:57.116945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:57.116945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:57.116945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:57.116945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:57.116945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:57.116945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:57.116945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:57.116945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:57.116945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:57.116945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:57.116945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:57.116945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:57.116945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:57.116945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:57.116945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:57.117707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:57.117707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:57.117707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:57.117707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:57.117707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:57.117707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:57.117707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:57.117707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:57.117707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:57.117707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:57.117707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:57.117707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:29:57.125335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 78/151] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:57.132059 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deff02a-8e14-4aa9-8cff-fbb79435f29e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:57.261993 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1002b-ebfe-467a-89ea-9032a58a8ed5 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.262647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1002b-ebfe-467a-89ea-9032a58a8ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.263123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1002b-ebfe-467a-89ea-9032a58a8ed5 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.265085 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 79/152] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:57.267540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:57.281717 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25009915-184a-47b1-a141-561d18ad34e8 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.282647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25009915-184a-47b1-a141-561d18ad34e8 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.283174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25009915-184a-47b1-a141-561d18ad34e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.303787 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deff02a-8e14-4aa9-8cff-fbb79435f29e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:57.312394 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-25009915-184a-47b1-a141-561d18ad34e8 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:57.314055 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-25009915-184a-47b1-a141-561d18ad34e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:57.370510 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:57.377162 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77a4fc-360a-40e2-99ed-3cee078868c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:57.378937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77a4fc-360a-40e2-99ed-3cee078868c7 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.379616 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77a4fc-360a-40e2-99ed-3cee078868c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.381450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77a4fc-360a-40e2-99ed-3cee078868c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.382466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77a4fc-360a-40e2-99ed-3cee078868c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:57.382966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77a4fc-360a-40e2-99ed-3cee078868c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:57.385488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77a4fc-360a-40e2-99ed-3cee078868c7 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:57.411527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 74/153] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:57.417655 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77a4fc-360a-40e2-99ed-3cee078868c7 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:57.462788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 80/154] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:57.574104 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1db82f-a56b-408f-bfab-0c65fdb387ce None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.575468 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1db82f-a56b-408f-bfab-0c65fdb387ce None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.575977 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1db82f-a56b-408f-bfab-0c65fdb387ce None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.577702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 81/155] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:57.588269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346808d4-7e7a-4a57-aeeb-ee9105bbeda7 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.589144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346808d4-7e7a-4a57-aeeb-ee9105bbeda7 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.590192 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346808d4-7e7a-4a57-aeeb-ee9105bbeda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.617044 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77a4fc-360a-40e2-99ed-3cee078868c7 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:57.623199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-346808d4-7e7a-4a57-aeeb-ee9105bbeda7 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:57.624815 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-346808d4-7e7a-4a57-aeeb-ee9105bbeda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:57.715934 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5851d3-6e71-4926-91ef-eed9baaeba05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:57.716692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5851d3-6e71-4926-91ef-eed9baaeba05 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.716966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5851d3-6e71-4926-91ef-eed9baaeba05 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.717233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5851d3-6e71-4926-91ef-eed9baaeba05 None admin] PATH_INFO: `/v3/users/7ded74de77c442b1853b2349fb3567ad` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.719063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5851d3-6e71-4926-91ef-eed9baaeba05 None admin] RBAC: Authorizing `identity:get_user(user_id=7ded74de77c442b1853b2349fb3567ad)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:57.720310 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5851d3-6e71-4926-91ef-eed9baaeba05 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:57.722550 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 75/156] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:57 2022] GET /identity/v3/users/7ded74de77c442b1853b2349fb3567ad => generated 275 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:57.730941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5851d3-6e71-4926-91ef-eed9baaeba05 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:57.741257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:57.744217 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 82/157] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:57.864098 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5851d3-6e71-4926-91ef-eed9baaeba05 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:57.911564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619b734-3812-47ac-9ecd-a2ffd13d845d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:57.912241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619b734-3812-47ac-9ecd-a2ffd13d845d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.912524 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619b734-3812-47ac-9ecd-a2ffd13d845d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.912784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619b734-3812-47ac-9ecd-a2ffd13d845d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.988822 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619b734-3812-47ac-9ecd-a2ffd13d845d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:57.993739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619b734-3812-47ac-9ecd-a2ffd13d845d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:58.003119 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:58.039097 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346808d4-7e7a-4a57-aeeb-ee9105bbeda7 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:58.142065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 76/158] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:58.152242 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619b734-3812-47ac-9ecd-a2ffd13d845d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:58.197570 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:58.216586 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346808d4-7e7a-4a57-aeeb-ee9105bbeda7 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:58.272065 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95525968-6733-4f68-82c8-250dbfbad398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:58.273126 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95525968-6733-4f68-82c8-250dbfbad398 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:58.273788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95525968-6733-4f68-82c8-250dbfbad398 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:58.274969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95525968-6733-4f68-82c8-250dbfbad398 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:58.305651 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95525968-6733-4f68-82c8-250dbfbad398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:58.319960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-95525968-6733-4f68-82c8-250dbfbad398 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:58.319960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:58.319960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:58.319960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:58.319960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:58.319960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:58.319960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:58.319960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:58.319960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:58.319960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:58.319960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:58.319960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:58.319960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:58.319960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:58.319960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:58.319960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:58.320996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:58.320996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:58.320996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:58.320996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:58.320996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:58.320996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:58.320996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:58.320996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:58.320996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:58.320996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:58.320996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:58.320996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:58.320996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:58.320996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:58.320996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:58.320996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:58.321807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:58.321807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:58.321807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:58.321807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:58.321807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:58.321807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:58.321807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:58.321807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:58.321807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:58.321807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:58.321807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:58.321807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:29:58.330826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 83/159] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:29:58.337171 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619b734-3812-47ac-9ecd-a2ffd13d845d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:58.338415 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95525968-6733-4f68-82c8-250dbfbad398 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:58.363719 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:58.431039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce74aed-55ec-44dd-b6c2-85f4995cfbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:58.432109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce74aed-55ec-44dd-b6c2-85f4995cfbeb None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:58.432606 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce74aed-55ec-44dd-b6c2-85f4995cfbeb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:58.433062 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce74aed-55ec-44dd-b6c2-85f4995cfbeb None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:58.434587 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce74aed-55ec-44dd-b6c2-85f4995cfbeb None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:58.436308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce74aed-55ec-44dd-b6c2-85f4995cfbeb None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:58.438256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 77/160] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:58 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:58.442497 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce74aed-55ec-44dd-b6c2-85f4995cfbeb None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:58.573652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95525968-6733-4f68-82c8-250dbfbad398 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:58.677927 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34722723-a8a3-447b-a0d5-a76b0baa23bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:58.678580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34722723-a8a3-447b-a0d5-a76b0baa23bc None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:58.678858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34722723-a8a3-447b-a0d5-a76b0baa23bc None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:58.679130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34722723-a8a3-447b-a0d5-a76b0baa23bc None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:58.679849 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34722723-a8a3-447b-a0d5-a76b0baa23bc None admin] RBAC: Adding query filter params (name=member) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:58.680185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34722723-a8a3-447b-a0d5-a76b0baa23bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:58.681807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34722723-a8a3-447b-a0d5-a76b0baa23bc None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:58.700502 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 84/161] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:58.700713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce74aed-55ec-44dd-b6c2-85f4995cfbeb None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:58.701351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34722723-a8a3-447b-a0d5-a76b0baa23bc None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:58.734467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:58.808983 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985308f3-950d-4e00-b050-cc60fb097a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:58.809672 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985308f3-950d-4e00-b050-cc60fb097a7e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:58.809951 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985308f3-950d-4e00-b050-cc60fb097a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:58.810227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985308f3-950d-4e00-b050-cc60fb097a7e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:58.841160 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985308f3-950d-4e00-b050-cc60fb097a7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:58.846404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-985308f3-950d-4e00-b050-cc60fb097a7e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:58.846404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:58.846404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:58.846404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:58.846404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:58.846404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:58.846404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:58.846404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:58.846404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:58.846404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:58.846404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:58.846404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:58.846404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:58.846404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:58.846404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:58.846404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:58.847414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:58.847414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:58.847414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:58.847414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:58.847414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:58.847414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:58.847414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:58.847414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:58.847414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:58.847414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:58.847414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:58.847414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:58.847414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:58.847414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:58.847414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:58.847414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:58.848228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:58.848228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:58.848228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:58.848228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:58.848228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:58.848228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:58.848228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:58.848228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:58.848228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:58.848228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:58.848228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:58.848228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:29:58.848228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 78/162] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:29:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:58.849368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f126f-a2e5-4b92-baa5-84eed78aa5ad None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:58.849368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f126f-a2e5-4b92-baa5-84eed78aa5ad None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:58.856402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f126f-a2e5-4b92-baa5-84eed78aa5ad None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:58.858206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 79/163] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:58.859067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f126f-a2e5-4b92-baa5-84eed78aa5ad None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:58.942943 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:58.965481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34722723-a8a3-447b-a0d5-a76b0baa23bc None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:59.065267 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7185e5b-b613-4b27-b0f2-4c1804d03d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:59.066067 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7185e5b-b613-4b27-b0f2-4c1804d03d40 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.066281 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7185e5b-b613-4b27-b0f2-4c1804d03d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.066561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7185e5b-b613-4b27-b0f2-4c1804d03d40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.067431 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7185e5b-b613-4b27-b0f2-4c1804d03d40 None admin] RBAC: Adding query filter params (role.id=4eebd8098ab44bfc84cd3de29a679bdf, scope.project.id=e02f7e20d2404b3888cc2e57180a901b, user.id=7ded74de77c442b1853b2349fb3567ad) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:59.067745 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7185e5b-b613-4b27-b0f2-4c1804d03d40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:59.069426 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7185e5b-b613-4b27-b0f2-4c1804d03d40 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:59.086497 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 85/164] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:58 2022] GET /identity/v3/role_assignments?user.id=7ded74de77c442b1853b2349fb3567ad&scope.project.id=e02f7e20d2404b3888cc2e57180a901b&role.id=4eebd8098ab44bfc84cd3de29a679bdf => generated 608 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:59.087527 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7185e5b-b613-4b27-b0f2-4c1804d03d40 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:59.102456 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f126f-a2e5-4b92-baa5-84eed78aa5ad None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:59.183069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7810faa-31c4-4688-b61a-88bfd448303b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:59.183903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7810faa-31c4-4688-b61a-88bfd448303b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.183982 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7810faa-31c4-4688-b61a-88bfd448303b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.184272 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7810faa-31c4-4688-b61a-88bfd448303b None admin] PATH_INFO: `/v3/users/7ded74de77c442b1853b2349fb3567ad` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.190343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7810faa-31c4-4688-b61a-88bfd448303b None admin] RBAC: Authorizing `identity:get_user(user_id=7ded74de77c442b1853b2349fb3567ad)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:59.192487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7810faa-31c4-4688-b61a-88bfd448303b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:59.194346 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 80/165] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:58 2022] GET /identity/v3/users/7ded74de77c442b1853b2349fb3567ad => generated 275 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:59.196179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5b05c-e9bc-4a0e-a557-239ad4956e36 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.196179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5b05c-e9bc-4a0e-a557-239ad4956e36 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.196179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5b05c-e9bc-4a0e-a557-239ad4956e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.229374 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8fe5b05c-e9bc-4a0e-a557-239ad4956e36 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:59.230827 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8fe5b05c-e9bc-4a0e-a557-239ad4956e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:59.286931 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:59.340454 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7185e5b-b613-4b27-b0f2-4c1804d03d40 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:59.364879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 81/166] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:59.365710 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5b05c-e9bc-4a0e-a557-239ad4956e36 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:59.417489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51d8a3-934b-4a3b-9cb5-213a16d9a828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:59.419375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51d8a3-934b-4a3b-9cb5-213a16d9a828 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.419655 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51d8a3-934b-4a3b-9cb5-213a16d9a828 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.419946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51d8a3-934b-4a3b-9cb5-213a16d9a828 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.420670 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51d8a3-934b-4a3b-9cb5-213a16d9a828 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:59.421240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51d8a3-934b-4a3b-9cb5-213a16d9a828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:59.426013 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51d8a3-934b-4a3b-9cb5-213a16d9a828 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:59.447264 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:59.451520 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 86/167] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:59.461042 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51d8a3-934b-4a3b-9cb5-213a16d9a828 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:59.653697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5b05c-e9bc-4a0e-a557-239ad4956e36 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:59.707240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51d8a3-934b-4a3b-9cb5-213a16d9a828 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:59.710951 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd7315-50df-44e3-9fe2-86b68f4171e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:59.711889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd7315-50df-44e3-9fe2-86b68f4171e7 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.712454 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd7315-50df-44e3-9fe2-86b68f4171e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.712934 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd7315-50df-44e3-9fe2-86b68f4171e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.774132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fd80a-9ef3-425b-adfc-613a2fc9ec78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:59.774952 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fd80a-9ef3-425b-adfc-613a2fc9ec78 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.775298 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fd80a-9ef3-425b-adfc-613a2fc9ec78 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.776814 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fd80a-9ef3-425b-adfc-613a2fc9ec78 None admin] PATH_INFO: `/v3/users/7ded74de77c442b1853b2349fb3567ad` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.778362 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fd80a-9ef3-425b-adfc-613a2fc9ec78 None admin] RBAC: Authorizing `identity:get_user(user_id=7ded74de77c442b1853b2349fb3567ad)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:59.779972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fd80a-9ef3-425b-adfc-613a2fc9ec78 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:59.785798 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 87/168] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:59 2022] GET /identity/v3/users/7ded74de77c442b1853b2349fb3567ad => generated 275 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:59.785798 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f41a4-0540-43f9-a00c-a1ee7aaa8580 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.785798 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f41a4-0540-43f9-a00c-a1ee7aaa8580 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.785798 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f41a4-0540-43f9-a00c-a1ee7aaa8580 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.785798 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 88/169] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:59.793601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb89ab-e686-4656-9ec6-fee574d69c7b None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.793902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb89ab-e686-4656-9ec6-fee574d69c7b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.794190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb89ab-e686-4656-9ec6-fee574d69c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.815084 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:59.830103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-fffb89ab-e686-4656-9ec6-fee574d69c7b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:59.831377 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-fffb89ab-e686-4656-9ec6-fee574d69c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:59.836629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd7315-50df-44e3-9fe2-86b68f4171e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:59.838504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd7315-50df-44e3-9fe2-86b68f4171e7 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:59.889371 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:59.957418 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 89/170] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:59.958273 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb89ab-e686-4656-9ec6-fee574d69c7b None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:00.008851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 82/171] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:00.014586 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd7315-50df-44e3-9fe2-86b68f4171e7 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:00.101142 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb89ab-e686-4656-9ec6-fee574d69c7b None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:00.152856 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955d888-eb99-416c-a6b3-083f20298320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:00.153502 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955d888-eb99-416c-a6b3-083f20298320 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:00.153802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955d888-eb99-416c-a6b3-083f20298320 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:00.154040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955d888-eb99-416c-a6b3-083f20298320 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:00.158243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:00.218663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd7315-50df-44e3-9fe2-86b68f4171e7 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:00.305620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955d888-eb99-416c-a6b3-083f20298320 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:00.308511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955d888-eb99-416c-a6b3-083f20298320 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:00.332401 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:00.345533 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ec379-e603-487a-ad32-2f9f6041f4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:00.346843 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ec379-e603-487a-ad32-2f9f6041f4fa None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:00.347332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ec379-e603-487a-ad32-2f9f6041f4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:00.347925 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ec379-e603-487a-ad32-2f9f6041f4fa None admin] PATH_INFO: `/v3/projects/2ec1a8ad7bf7456ea8ef6d0d90bf47a7` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:00.351468 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ec379-e603-487a-ad32-2f9f6041f4fa None admin] RBAC: Authorizing `identity:get_project(project_id=2ec1a8ad7bf7456ea8ef6d0d90bf47a7)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:00.354738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ec379-e603-487a-ad32-2f9f6041f4fa None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:00.361570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 83/172] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:00 2022] GET /identity/v3/projects/2ec1a8ad7bf7456ea8ef6d0d90bf47a7 => generated 312 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:00.363087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ec379-e603-487a-ad32-2f9f6041f4fa None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:00.529043 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ec379-e603-487a-ad32-2f9f6041f4fa None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:00.554649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 90/173] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:00.556026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac03a72-74f6-4054-b3f9-32c143464590 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:00.556311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac03a72-74f6-4054-b3f9-32c143464590 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:00.556607 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac03a72-74f6-4054-b3f9-32c143464590 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:00.558107 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 91/174] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:00.558944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac03a72-74f6-4054-b3f9-32c143464590 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:00.586381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5c9ef-8aac-4b5b-bdaa-05c792e3c876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:00.587890 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5c9ef-8aac-4b5b-bdaa-05c792e3c876 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:00.588686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5c9ef-8aac-4b5b-bdaa-05c792e3c876 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:00.589272 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5c9ef-8aac-4b5b-bdaa-05c792e3c876 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:00.608680 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a5c9ef-8aac-4b5b-bdaa-05c792e3c876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:00.609560 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:00.617837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-20a5c9ef-8aac-4b5b-bdaa-05c792e3c876 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:00.617837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:00.617837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:00.617837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:00.617837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:00.617837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:00.617837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:00.617837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:00.617837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:00.617837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:00.617837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:00.617837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:00.617837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:00.617837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:00.617837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:00.617837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:00.618860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:00.618860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:00.618860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:00.618860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:00.618860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:00.618860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:00.618860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:00.618860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:00.618860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:00.618860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:00.618860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:00.618860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:00.618860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:00.618860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:00.618860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:00.618860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:00.619643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:00.619643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:00.619643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:00.619643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:00.619643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:00.619643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:00.619643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:00.619643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:00.619643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:00.619643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:00.619643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:00.619643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:00.629356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 84/175] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:00.629356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5c9ef-8aac-4b5b-bdaa-05c792e3c876 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:00.702184 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:00.725478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac03a72-74f6-4054-b3f9-32c143464590 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:00.753218 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5c9ef-8aac-4b5b-bdaa-05c792e3c876 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:00.786875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b69b7-25cb-48fd-b56d-0a4e7c65633c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:00.787840 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b69b7-25cb-48fd-b56d-0a4e7c65633c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:00.788426 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b69b7-25cb-48fd-b56d-0a4e7c65633c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:00.788877 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b69b7-25cb-48fd-b56d-0a4e7c65633c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:00.790009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ddc60-c300-45e3-a6c7-37dac9ff6ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:00.790230 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b69b7-25cb-48fd-b56d-0a4e7c65633c None admin] RBAC: Adding query filter params (role.id=4eebd8098ab44bfc84cd3de29a679bdf, scope.project.id=2ec1a8ad7bf7456ea8ef6d0d90bf47a7, user.id=7ded74de77c442b1853b2349fb3567ad) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:00.790658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ddc60-c300-45e3-a6c7-37dac9ff6ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:00.790788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b69b7-25cb-48fd-b56d-0a4e7c65633c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:00.790915 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ddc60-c300-45e3-a6c7-37dac9ff6ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:00.791188 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ddc60-c300-45e3-a6c7-37dac9ff6ee8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:00.793527 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b69b7-25cb-48fd-b56d-0a4e7c65633c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:00.804264 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ddc60-c300-45e3-a6c7-37dac9ff6ee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:00.804558 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 92/176] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:00 2022] GET /identity/v3/role_assignments?user.id=7ded74de77c442b1853b2349fb3567ad&scope.project.id=2ec1a8ad7bf7456ea8ef6d0d90bf47a7&role.id=4eebd8098ab44bfc84cd3de29a679bdf => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:00.806739 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea08ee-80bd-47fa-b16d-9df171e12ae1 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:00.807218 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea08ee-80bd-47fa-b16d-9df171e12ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:00.807692 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea08ee-80bd-47fa-b16d-9df171e12ae1 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:00.809216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-869ddc60-c300-45e3-a6c7-37dac9ff6ee8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:00.809216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:00.809216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:00.809216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:00.809216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:00.809216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:00.809216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:00.809216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:00.809216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:00.809216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:00.809216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:00.809216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:00.809216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:00.809216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:00.809216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:00.809216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:00.810745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:00.810745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:00.810745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:00.810745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:00.810745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:00.810745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:00.810745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:00.810745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:00.810745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:00.810745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:00.810745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:00.810745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:00.810745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:00.810745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:00.810745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:00.810745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:00.812248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:00.812248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:00.812248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:00.812248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:00.812248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:00.812248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:00.812248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:00.812248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:00.812248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:00.812248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:00.812248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:00.812248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:00.813407 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 93/177] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:00.813548 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 85/178] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:00.813548 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ddc60-c300-45e3-a6c7-37dac9ff6ee8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:00.813758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496f08b-cdc9-4522-8408-641df6d6a5b1 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:00.813758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496f08b-cdc9-4522-8408-641df6d6a5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:00.813758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496f08b-cdc9-4522-8408-641df6d6a5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:00.849547 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-f496f08b-cdc9-4522-8408-641df6d6a5b1 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:00.851512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f496f08b-cdc9-4522-8408-641df6d6a5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:00.980072 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 94/179] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:00.980072 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496f08b-cdc9-4522-8408-641df6d6a5b1 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:01.008666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ddc60-c300-45e3-a6c7-37dac9ff6ee8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.035809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:01.055223 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:01.092141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba33f65-21ec-4b54-88fe-a9421be78775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:01.093213 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba33f65-21ec-4b54-88fe-a9421be78775 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.093702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba33f65-21ec-4b54-88fe-a9421be78775 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.094172 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba33f65-21ec-4b54-88fe-a9421be78775 None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.096422 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba33f65-21ec-4b54-88fe-a9421be78775 None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:01.099352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba33f65-21ec-4b54-88fe-a9421be78775 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:01.102646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 86/180] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:00 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:01.108063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e985d-28e7-413e-b194-5996d94f780d None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.108063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e985d-28e7-413e-b194-5996d94f780d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.108063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e985d-28e7-413e-b194-5996d94f780d None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.108539 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 87/181] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:01.110714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591fa00-309c-439e-adab-68caf96da5eb None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.111209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591fa00-309c-439e-adab-68caf96da5eb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.111738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591fa00-309c-439e-adab-68caf96da5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.146066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-b591fa00-309c-439e-adab-68caf96da5eb None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:01.148388 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b591fa00-309c-439e-adab-68caf96da5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:01.170511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496f08b-cdc9-4522-8408-641df6d6a5b1 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.224747 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e570e43-9ef7-4929-942d-0150222fcad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:01.226805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e570e43-9ef7-4929-942d-0150222fcad8 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.227808 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e570e43-9ef7-4929-942d-0150222fcad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.228495 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e570e43-9ef7-4929-942d-0150222fcad8 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.229907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e570e43-9ef7-4929-942d-0150222fcad8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:01.230640 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e570e43-9ef7-4929-942d-0150222fcad8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:01.233479 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e570e43-9ef7-4929-942d-0150222fcad8 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:01.247835 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 88/182] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:01.251466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 95/183] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:01.253284 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e570e43-9ef7-4929-942d-0150222fcad8 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:01.257151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591fa00-309c-439e-adab-68caf96da5eb None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:01.403090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:01.423730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e570e43-9ef7-4929-942d-0150222fcad8 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.435464 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591fa00-309c-439e-adab-68caf96da5eb None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.462964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:01.487263 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645f735-b31e-4915-b1d6-95119f5ad7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:01.488229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645f735-b31e-4915-b1d6-95119f5ad7d3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.488569 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645f735-b31e-4915-b1d6-95119f5ad7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.488848 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645f735-b31e-4915-b1d6-95119f5ad7d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.489554 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645f735-b31e-4915-b1d6-95119f5ad7d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:01.489860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645f735-b31e-4915-b1d6-95119f5ad7d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:01.491292 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645f735-b31e-4915-b1d6-95119f5ad7d3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:01.495539 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39fb02-1f36-4882-9719-e4036f83512e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:01.496645 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39fb02-1f36-4882-9719-e4036f83512e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.497125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39fb02-1f36-4882-9719-e4036f83512e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.498421 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39fb02-1f36-4882-9719-e4036f83512e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.499875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39fb02-1f36-4882-9719-e4036f83512e None admin] RBAC: Adding query filter params (role.id=aedf4e1e5c7848099d7bd183778f5896, scope.project.id=e02f7e20d2404b3888cc2e57180a901b, user.id=7ded74de77c442b1853b2349fb3567ad) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:01.500405 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39fb02-1f36-4882-9719-e4036f83512e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:01.509383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 89/184] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:01.509383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25528982-fd9b-4586-be83-e6b19f8061d2 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.509383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25528982-fd9b-4586-be83-e6b19f8061d2 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.509890 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39fb02-1f36-4882-9719-e4036f83512e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:01.510072 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25528982-fd9b-4586-be83-e6b19f8061d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.515307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 96/185] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:01 2022] GET /identity/v3/role_assignments?user.id=7ded74de77c442b1853b2349fb3567ad&scope.project.id=e02f7e20d2404b3888cc2e57180a901b&role.id=aedf4e1e5c7848099d7bd183778f5896 => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:01.516943 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39fb02-1f36-4882-9719-e4036f83512e None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:01.534782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-25528982-fd9b-4586-be83-e6b19f8061d2 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:01.537379 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-25528982-fd9b-4586-be83-e6b19f8061d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:01.649034 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 90/186] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:01.649712 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25528982-fd9b-4586-be83-e6b19f8061d2 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:01.737945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39fb02-1f36-4882-9719-e4036f83512e None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.796835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27eb89-8e86-4d5b-9a14-4874e7827212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:01.797580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27eb89-8e86-4d5b-9a14-4874e7827212 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.798527 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27eb89-8e86-4d5b-9a14-4874e7827212 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.798841 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27eb89-8e86-4d5b-9a14-4874e7827212 None admin] PATH_INFO: `/v3/users/5f1827dde46f4915bfe3277a281b43bd` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.800543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c27eb89-8e86-4d5b-9a14-4874e7827212 None admin] RBAC: Authorizing `identity:get_user(user_id=5f1827dde46f4915bfe3277a281b43bd)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:01.802054 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c27eb89-8e86-4d5b-9a14-4874e7827212 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:01.803979 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 97/187] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:01 2022] GET /identity/v3/users/5f1827dde46f4915bfe3277a281b43bd => generated 247 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:01.805373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27eb89-8e86-4d5b-9a14-4874e7827212 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:01.815716 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25528982-fd9b-4586-be83-e6b19f8061d2 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.830971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:01.839527 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:01.869308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c27a5-9166-4a51-94af-761cc68dba1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:01.870363 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c27a5-9166-4a51-94af-761cc68dba1f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.870820 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c27a5-9166-4a51-94af-761cc68dba1f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.871245 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c27a5-9166-4a51-94af-761cc68dba1f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.873135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c27a5-9166-4a51-94af-761cc68dba1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:01.876048 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c27a5-9166-4a51-94af-761cc68dba1f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:01.879388 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 91/188] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:01.880754 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c27a5-9166-4a51-94af-761cc68dba1f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:01.932624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27eb89-8e86-4d5b-9a14-4874e7827212 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.987500 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce0b27-af4f-443a-921a-49d93f668430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:01.988671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce0b27-af4f-443a-921a-49d93f668430 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.989123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce0b27-af4f-443a-921a-49d93f668430 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.989625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce0b27-af4f-443a-921a-49d93f668430 None admin] PATH_INFO: `/v3/users/438d211883f446219e02376a2f35daa2` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:02.014125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce0b27-af4f-443a-921a-49d93f668430 None admin] RBAC: Authorizing `identity:get_user(user_id=438d211883f446219e02376a2f35daa2)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:02.017026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce0b27-af4f-443a-921a-49d93f668430 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:02.019302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c27a5-9166-4a51-94af-761cc68dba1f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:02.020361 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 98/189] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:01 2022] GET /identity/v3/users/438d211883f446219e02376a2f35daa2 => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:02.029356 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce0b27-af4f-443a-921a-49d93f668430 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:02.061534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f2c53-85fe-4ba5-8c03-132586169b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:02.062201 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f2c53-85fe-4ba5-8c03-132586169b4c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:02.062478 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f2c53-85fe-4ba5-8c03-132586169b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:02.062758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f2c53-85fe-4ba5-8c03-132586169b4c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:02.076485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f2c53-85fe-4ba5-8c03-132586169b4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:02.081528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-bf1f2c53-85fe-4ba5-8c03-132586169b4c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:30:02.081528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:02.081528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:02.081528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:02.081528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:02.081528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:02.081528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:02.081528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:02.081528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:02.081528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:02.081528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:02.081528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:02.081528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:02.081528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:02.081528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:02.081528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:02.082625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:02.082625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:02.082625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:02.082625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:02.082625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:02.082625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:02.082625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:02.082625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:02.082625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:02.082625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:02.082625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:02.082625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:02.082625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:02.082625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:02.082625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:02.082625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:02.083419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:02.083419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:02.083419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:02.083419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:02.083419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:02.083419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:02.083419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:02.083419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:02.083419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:02.083419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:02.083419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:30:02.083419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:02.083419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 92/190] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:02.084492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f2c53-85fe-4ba5-8c03-132586169b4c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:02.133476 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:02.155140 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:02.181207 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce0b27-af4f-443a-921a-49d93f668430 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:02.203613 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f2c53-85fe-4ba5-8c03-132586169b4c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:02.227385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6487336-b5c0-4e9f-9da4-d398d8a182b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:02.228465 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6487336-b5c0-4e9f-9da4-d398d8a182b6 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:02.228953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6487336-b5c0-4e9f-9da4-d398d8a182b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:02.229424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6487336-b5c0-4e9f-9da4-d398d8a182b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:02.244331 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65078063-9f7c-43cc-b0fd-8ce86af1e2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:02.245006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65078063-9f7c-43cc-b0fd-8ce86af1e2a6 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:02.245481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65078063-9f7c-43cc-b0fd-8ce86af1e2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:02.245840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65078063-9f7c-43cc-b0fd-8ce86af1e2a6 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:02.246737 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65078063-9f7c-43cc-b0fd-8ce86af1e2a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:02.248611 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65078063-9f7c-43cc-b0fd-8ce86af1e2a6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:02.280095 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 93/191] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:30:02 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:02.282051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65078063-9f7c-43cc-b0fd-8ce86af1e2a6 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:02.326568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6487336-b5c0-4e9f-9da4-d398d8a182b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:02.329818 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6487336-b5c0-4e9f-9da4-d398d8a182b6 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:02.435756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:02.462569 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:02.465071 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 99/192] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:02.466173 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6487336-b5c0-4e9f-9da4-d398d8a182b6 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:02.474620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65078063-9f7c-43cc-b0fd-8ce86af1e2a6 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:02.539713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a442b4-34f8-418a-99d0-a9e115e13809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:02.540646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a442b4-34f8-418a-99d0-a9e115e13809 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:02.541231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a442b4-34f8-418a-99d0-a9e115e13809 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:02.541654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a442b4-34f8-418a-99d0-a9e115e13809 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:02.648259 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6487336-b5c0-4e9f-9da4-d398d8a182b6 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:02.675241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a442b4-34f8-418a-99d0-a9e115e13809 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:02.677295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a442b4-34f8-418a-99d0-a9e115e13809 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:02.728820 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c09fb-978f-45d8-801f-4db52844a51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:02.729846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c09fb-978f-45d8-801f-4db52844a51d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:02.730306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c09fb-978f-45d8-801f-4db52844a51d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:02.730788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c09fb-978f-45d8-801f-4db52844a51d None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:02.732036 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c09fb-978f-45d8-801f-4db52844a51d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:02.732561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c09fb-978f-45d8-801f-4db52844a51d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:02.735211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c09fb-978f-45d8-801f-4db52844a51d None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:02.752215 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 100/193] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:02.753898 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c09fb-978f-45d8-801f-4db52844a51d None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:02.801899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:02.812691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 94/194] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:02.820170 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a442b4-34f8-418a-99d0-a9e115e13809 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:02.842165 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:02.917713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c09fb-978f-45d8-801f-4db52844a51d None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:02.960471 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea2e82-d7a2-42eb-8cf4-e285ab09fb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:02.961130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea2e82-d7a2-42eb-8cf4-e285ab09fb78 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:02.961409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea2e82-d7a2-42eb-8cf4-e285ab09fb78 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:02.961681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea2e82-d7a2-42eb-8cf4-e285ab09fb78 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:02.974141 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea2e82-d7a2-42eb-8cf4-e285ab09fb78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:02.979542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-17ea2e82-d7a2-42eb-8cf4-e285ab09fb78 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:02.979542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:02.979542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:02.979542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:02.979542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:02.979542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:02.979542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:02.979542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:02.979542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:02.979542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:02.979542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:02.979542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:02.979542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:02.979542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:02.979542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:02.979542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:02.980718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:02.980718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:02.980718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:02.980718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:02.980718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:02.980718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:02.980718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:02.980718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:02.980718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:02.980718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:02.980718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:02.980718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:02.980718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:02.980718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:02.980718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:02.980718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:02.981662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:02.981662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:02.981662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:02.981662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:02.981662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:02.981662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:02.981662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:02.981662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:02.981662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:02.981662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:02.981662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:02.981662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:02.981662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 101/195] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:02.981662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea2e82-d7a2-42eb-8cf4-e285ab09fb78 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:02.984719 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a442b4-34f8-418a-99d0-a9e115e13809 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.028672 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3edab-5b67-40ee-8117-456cf316fc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.029600 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3edab-5b67-40ee-8117-456cf316fc54 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.029884 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3edab-5b67-40ee-8117-456cf316fc54 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.030157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3edab-5b67-40ee-8117-456cf316fc54 None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.031346 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3edab-5b67-40ee-8117-456cf316fc54 None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.033454 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3edab-5b67-40ee-8117-456cf316fc54 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.036419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 95/196] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:02 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:03.037306 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3edab-5b67-40ee-8117-456cf316fc54 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.118962 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea2e82-d7a2-42eb-8cf4-e285ab09fb78 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.122767 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:03.131287 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:03.161736 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab09eed-ed7c-468d-abb2-29da270b1403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.162571 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab09eed-ed7c-468d-abb2-29da270b1403 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.162939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab09eed-ed7c-468d-abb2-29da270b1403 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.163338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab09eed-ed7c-468d-abb2-29da270b1403 None admin] PATH_INFO: `/v3/users/b9690d46a2454005a20d429acb3d3525` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.164651 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab09eed-ed7c-468d-abb2-29da270b1403 None admin] RBAC: Authorizing `identity:get_user(user_id=b9690d46a2454005a20d429acb3d3525)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.166269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab09eed-ed7c-468d-abb2-29da270b1403 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.172783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 102/197] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:02 2022] GET /identity/v3/users/b9690d46a2454005a20d429acb3d3525 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:03.172783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab09eed-ed7c-468d-abb2-29da270b1403 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.195191 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3edab-5b67-40ee-8117-456cf316fc54 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.259868 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88898f5-650b-41ed-ac36-b329c3f01be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.260567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88898f5-650b-41ed-ac36-b329c3f01be4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.260851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88898f5-650b-41ed-ac36-b329c3f01be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.261180 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88898f5-650b-41ed-ac36-b329c3f01be4 None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.262425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88898f5-650b-41ed-ac36-b329c3f01be4 None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.265008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88898f5-650b-41ed-ac36-b329c3f01be4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.265651 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 96/198] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:03.266645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88898f5-650b-41ed-ac36-b329c3f01be4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.300777 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab09eed-ed7c-468d-abb2-29da270b1403 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.340423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21004ec9-4f50-45d4-b80d-afa838e83fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.341151 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21004ec9-4f50-45d4-b80d-afa838e83fba None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.341521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21004ec9-4f50-45d4-b80d-afa838e83fba None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.341795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21004ec9-4f50-45d4-b80d-afa838e83fba None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.342480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21004ec9-4f50-45d4-b80d-afa838e83fba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:03.342776 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21004ec9-4f50-45d4-b80d-afa838e83fba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.344391 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21004ec9-4f50-45d4-b80d-afa838e83fba None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.354586 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 103/199] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:03.355317 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21004ec9-4f50-45d4-b80d-afa838e83fba None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.397465 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88898f5-650b-41ed-ac36-b329c3f01be4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.397813 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:03.431874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:03.438853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e14bfd-fdc1-4038-ae8b-6e235510ab64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.439964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e14bfd-fdc1-4038-ae8b-6e235510ab64 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.441223 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e14bfd-fdc1-4038-ae8b-6e235510ab64 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.441223 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e14bfd-fdc1-4038-ae8b-6e235510ab64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.441985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e14bfd-fdc1-4038-ae8b-6e235510ab64 None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=e02f7e20d2404b3888cc2e57180a901b, user.id=5f1827dde46f4915bfe3277a281b43bd) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:03.442362 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e14bfd-fdc1-4038-ae8b-6e235510ab64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.443962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e14bfd-fdc1-4038-ae8b-6e235510ab64 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.456032 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 97/200] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/role_assignments?user.id=5f1827dde46f4915bfe3277a281b43bd&scope.project.id=e02f7e20d2404b3888cc2e57180a901b&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:03.456032 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e14bfd-fdc1-4038-ae8b-6e235510ab64 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.502648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21004ec9-4f50-45d4-b80d-afa838e83fba None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.569827 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160888a4-a70f-4a01-8f62-cfa34193c3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.570817 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160888a4-a70f-4a01-8f62-cfa34193c3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.571264 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160888a4-a70f-4a01-8f62-cfa34193c3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.571706 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160888a4-a70f-4a01-8f62-cfa34193c3c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.573811 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e14bfd-fdc1-4038-ae8b-6e235510ab64 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.645229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7084436-b9af-4239-bf4d-e150b75118ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.645946 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7084436-b9af-4239-bf4d-e150b75118ad None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.646232 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7084436-b9af-4239-bf4d-e150b75118ad None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.646503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7084436-b9af-4239-bf4d-e150b75118ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.647301 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7084436-b9af-4239-bf4d-e150b75118ad None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=b994c620a91d441aa453e5be1b958864, user.id=438d211883f446219e02376a2f35daa2) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:03.647674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7084436-b9af-4239-bf4d-e150b75118ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.649412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7084436-b9af-4239-bf4d-e150b75118ad None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.664753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 98/201] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/role_assignments?user.id=438d211883f446219e02376a2f35daa2&scope.project.id=b994c620a91d441aa453e5be1b958864&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:03.665666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7084436-b9af-4239-bf4d-e150b75118ad None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.672862 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160888a4-a70f-4a01-8f62-cfa34193c3c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.675005 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160888a4-a70f-4a01-8f62-cfa34193c3c8 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.718901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:03.784570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:03.808200 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 104/202] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:03.814941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160888a4-a70f-4a01-8f62-cfa34193c3c8 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.877297 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7084436-b9af-4239-bf4d-e150b75118ad None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.946922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7aaec-39d0-43f9-a481-2bf7beb67ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.947745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7aaec-39d0-43f9-a481-2bf7beb67ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.948041 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7aaec-39d0-43f9-a481-2bf7beb67ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.948317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7aaec-39d0-43f9-a481-2bf7beb67ad8 None admin] PATH_INFO: `/v3/users/438d211883f446219e02376a2f35daa2` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.952087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c7aaec-39d0-43f9-a481-2bf7beb67ad8 None admin] RBAC: Authorizing `identity:get_user(user_id=438d211883f446219e02376a2f35daa2)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.952087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c7aaec-39d0-43f9-a481-2bf7beb67ad8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.952933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 99/203] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/users/438d211883f446219e02376a2f35daa2 => generated 283 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:03.959831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7aaec-39d0-43f9-a481-2bf7beb67ad8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:04.036650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160888a4-a70f-4a01-8f62-cfa34193c3c8 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:04.110560 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a48c5-ba87-4a50-99c0-90bfe35edf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:04.111294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a48c5-ba87-4a50-99c0-90bfe35edf7c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:04.111573 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a48c5-ba87-4a50-99c0-90bfe35edf7c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:04.111839 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a48c5-ba87-4a50-99c0-90bfe35edf7c None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:04.113076 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a48c5-ba87-4a50-99c0-90bfe35edf7c None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:04.115062 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a48c5-ba87-4a50-99c0-90bfe35edf7c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:04.118914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 105/204] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:04.127482 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a48c5-ba87-4a50-99c0-90bfe35edf7c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:04.131129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:04.262403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a48c5-ba87-4a50-99c0-90bfe35edf7c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:04.271876 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7aaec-39d0-43f9-a481-2bf7beb67ad8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:04.308328 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d490d0-3e81-464d-974a-eedd524a8f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:04.308961 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d490d0-3e81-464d-974a-eedd524a8f9f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:04.309245 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d490d0-3e81-464d-974a-eedd524a8f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:04.309531 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d490d0-3e81-464d-974a-eedd524a8f9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:04.310274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d490d0-3e81-464d-974a-eedd524a8f9f None admin] RBAC: Adding query filter params (role.id=89f1b9db96764371860f845c00419e7a, scope.project.id=e02f7e20d2404b3888cc2e57180a901b, user.id=b9690d46a2454005a20d429acb3d3525) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:04.310594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d490d0-3e81-464d-974a-eedd524a8f9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:04.312072 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d490d0-3e81-464d-974a-eedd524a8f9f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:04.327477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:04.330135 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 106/205] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:04 2022] GET /identity/v3/role_assignments?user.id=b9690d46a2454005a20d429acb3d3525&scope.project.id=e02f7e20d2404b3888cc2e57180a901b&role.id=89f1b9db96764371860f845c00419e7a => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:04.342943 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04896f9d-d62a-420f-82ff-277fc278c47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:04.343919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04896f9d-d62a-420f-82ff-277fc278c47b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:04.348259 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04896f9d-d62a-420f-82ff-277fc278c47b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:04.348737 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04896f9d-d62a-420f-82ff-277fc278c47b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:04.473472 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04896f9d-d62a-420f-82ff-277fc278c47b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:04.475054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04896f9d-d62a-420f-82ff-277fc278c47b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:04.639058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 100/206] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 680 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:04.645297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d490d0-3e81-464d-974a-eedd524a8f9f None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:04.705517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205e5ca-0469-4449-8d29-ac6bd90aedac None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:04.706656 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205e5ca-0469-4449-8d29-ac6bd90aedac None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:04.707505 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205e5ca-0469-4449-8d29-ac6bd90aedac None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:04.711154 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 101/207] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:04.718700 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5454750-819d-4428-a1dc-22754c8086fb None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:04.719923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5454750-819d-4428-a1dc-22754c8086fb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:04.720839 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5454750-819d-4428-a1dc-22754c8086fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:04.754604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a5454750-819d-4428-a1dc-22754c8086fb None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:04.757090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a5454750-819d-4428-a1dc-22754c8086fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:04.785983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:04.821083 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:04.858632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d490d0-3e81-464d-974a-eedd524a8f9f None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:04.921648 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 102/208] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:04.927070 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb341b81-27c3-4466-bb39-c78788af645b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:04.927306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb341b81-27c3-4466-bb39-c78788af645b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:04.927598 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb341b81-27c3-4466-bb39-c78788af645b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:04.927839 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb341b81-27c3-4466-bb39-c78788af645b None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:04.931616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb341b81-27c3-4466-bb39-c78788af645b None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:04.934809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb341b81-27c3-4466-bb39-c78788af645b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:04.938591 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 107/209] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:04 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:04.945623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5454750-819d-4428-a1dc-22754c8086fb None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:05.101121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5454750-819d-4428-a1dc-22754c8086fb None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:05.144253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b2c83-5929-4622-b1f2-10cd53188608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:05.144897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b2c83-5929-4622-b1f2-10cd53188608 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:05.145172 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b2c83-5929-4622-b1f2-10cd53188608 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:05.145443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b2c83-5929-4622-b1f2-10cd53188608 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:05.146211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b2c83-5929-4622-b1f2-10cd53188608 None admin] RBAC: Adding query filter params (role.id=aedf4e1e5c7848099d7bd183778f5896, scope.project.id=b994c620a91d441aa453e5be1b958864, user.id=438d211883f446219e02376a2f35daa2) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:05.146517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b2c83-5929-4622-b1f2-10cd53188608 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:05.148201 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b2c83-5929-4622-b1f2-10cd53188608 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:05.155188 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 103/210] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:04 2022] GET /identity/v3/role_assignments?user.id=438d211883f446219e02376a2f35daa2&scope.project.id=b994c620a91d441aa453e5be1b958864&role.id=aedf4e1e5c7848099d7bd183778f5896 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:05.195242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb341b81-27c3-4466-bb39-c78788af645b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:05.359677 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb341b81-27c3-4466-bb39-c78788af645b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:05.398622 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b8a42-6050-4709-9606-34d27dd722a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:05.399276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b8a42-6050-4709-9606-34d27dd722a5 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:05.399551 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b8a42-6050-4709-9606-34d27dd722a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:05.399820 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b8a42-6050-4709-9606-34d27dd722a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:05.402022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:05.417429 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b8a42-6050-4709-9606-34d27dd722a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:05.426702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-187b8a42-6050-4709-9606-34d27dd722a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:05.426702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:05.426702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:05.426702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:05.426702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:05.426702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:05.426702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:05.426702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:05.426702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:05.426702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:05.426702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:05.426702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:05.426702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:05.426702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:05.426702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:05.426702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:05.427681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:05.427681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:05.427681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:05.427681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:05.427681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:05.427681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:05.427681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:05.427681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:05.427681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:05.427681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:05.427681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:05.427681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:05.427681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:05.427681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:05.427681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:05.427681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:05.428517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:05.428517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:05.428517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:05.428517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:05.428517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:05.428517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:05.428517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:05.428517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:05.428517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:05.428517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:05.428517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:05.428517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:05.430425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 108/211] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:05.434775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b2c83-5929-4622-b1f2-10cd53188608 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:05.462962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:05.475567 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ca2e6-e367-4f08-8a20-25492e3ff16d None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:05.475567 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ca2e6-e367-4f08-8a20-25492e3ff16d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:05.475567 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ca2e6-e367-4f08-8a20-25492e3ff16d None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:05.478588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 109/212] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:05.488026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0df04-7e96-403a-9662-29054946b2f9 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:05.488026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0df04-7e96-403a-9662-29054946b2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:05.488026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0df04-7e96-403a-9662-29054946b2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:05.520507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-43c0df04-7e96-403a-9662-29054946b2f9 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:05.527208 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-43c0df04-7e96-403a-9662-29054946b2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:05.595786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b2c83-5929-4622-b1f2-10cd53188608 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:05.642877 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb54d88-360c-4254-af13-7896618b2048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:05.643771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb54d88-360c-4254-af13-7896618b2048 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:05.644233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb54d88-360c-4254-af13-7896618b2048 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:05.644642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb54d88-360c-4254-af13-7896618b2048 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:05.645636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb54d88-360c-4254-af13-7896618b2048 None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:05.646105 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb54d88-360c-4254-af13-7896618b2048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:05.647984 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb54d88-360c-4254-af13-7896618b2048 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:05.658383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 104/213] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:05.667317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb54d88-360c-4254-af13-7896618b2048 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:05.672114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 110/214] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:05.825181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:05.850485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb54d88-360c-4254-af13-7896618b2048 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:05.924161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea68175-14c7-42fa-b49d-cff3f6efcc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:05.925234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea68175-14c7-42fa-b49d-cff3f6efcc03 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:05.925719 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea68175-14c7-42fa-b49d-cff3f6efcc03 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:05.926189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea68175-14c7-42fa-b49d-cff3f6efcc03 None admin] PATH_INFO: `/v3/users/5b475a64338649da996a33f42448bb5b` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:05.945354 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea68175-14c7-42fa-b49d-cff3f6efcc03 None admin] RBAC: Authorizing `identity:get_user(user_id=5b475a64338649da996a33f42448bb5b)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:05.947353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea68175-14c7-42fa-b49d-cff3f6efcc03 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:05.949382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 105/215] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:05 2022] GET /identity/v3/users/5b475a64338649da996a33f42448bb5b => generated 297 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:05.958280 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea68175-14c7-42fa-b49d-cff3f6efcc03 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:05.959648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a8231-73fd-4316-b9f1-f79fe3bbfb12 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:05.960330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a8231-73fd-4316-b9f1-f79fe3bbfb12 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:05.960808 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a8231-73fd-4316-b9f1-f79fe3bbfb12 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:05.963473 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 111/216] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:05.972302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bba53-14df-4af2-95af-46686ab4ca6c None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:05.972750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bba53-14df-4af2-95af-46686ab4ca6c None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:05.973233 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bba53-14df-4af2-95af-46686ab4ca6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:06.004013 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-2d8bba53-14df-4af2-95af-46686ab4ca6c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:06.006130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2d8bba53-14df-4af2-95af-46686ab4ca6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:06.139839 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 112/217] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:30:06.159827 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea68175-14c7-42fa-b49d-cff3f6efcc03 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:06.180183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bba53-14df-4af2-95af-46686ab4ca6c None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:06.228033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf3ae7-5045-41cd-a08a-0e78b2a3a93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:06.229231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf3ae7-5045-41cd-a08a-0e78b2a3a93b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:06.229712 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf3ae7-5045-41cd-a08a-0e78b2a3a93b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:06.230178 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf3ae7-5045-41cd-a08a-0e78b2a3a93b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:06.251942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:06.308289 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:06.342268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bba53-14df-4af2-95af-46686ab4ca6c None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:06.365316 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf3ae7-5045-41cd-a08a-0e78b2a3a93b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:06.368198 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf3ae7-5045-41cd-a08a-0e78b2a3a93b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:06.383008 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c274da-1172-4733-95ee-d83b5bf32710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:06.384526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c274da-1172-4733-95ee-d83b5bf32710 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:06.385100 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c274da-1172-4733-95ee-d83b5bf32710 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:06.385567 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c274da-1172-4733-95ee-d83b5bf32710 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:06.387020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c274da-1172-4733-95ee-d83b5bf32710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:06.388798 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c274da-1172-4733-95ee-d83b5bf32710 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:06.391056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 113/218] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:06.399934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c274da-1172-4733-95ee-d83b5bf32710 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:06.539710 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 106/219] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:06.548312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf3ae7-5045-41cd-a08a-0e78b2a3a93b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:06.606931 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:06.631598 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c274da-1172-4733-95ee-d83b5bf32710 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:06.680497 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:06.705021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf3ae7-5045-41cd-a08a-0e78b2a3a93b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:06.708648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618134e-44e9-457f-a1b0-4c24b046e7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:06.709653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618134e-44e9-457f-a1b0-4c24b046e7a2 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:06.709942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618134e-44e9-457f-a1b0-4c24b046e7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:06.710170 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618134e-44e9-457f-a1b0-4c24b046e7a2 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:06.710847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618134e-44e9-457f-a1b0-4c24b046e7a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:06.712572 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618134e-44e9-457f-a1b0-4c24b046e7a2 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:06.740350 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 114/220] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:30:06 2022] POST /identity/v3/users => generated 297 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:30:06.741337 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618134e-44e9-457f-a1b0-4c24b046e7a2 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:06.794173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1f291-7418-4c19-92b5-f5860b3af647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:06.795730 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1f291-7418-4c19-92b5-f5860b3af647 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:06.796548 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1f291-7418-4c19-92b5-f5860b3af647 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:06.797328 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1f291-7418-4c19-92b5-f5860b3af647 None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:06.800892 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1f291-7418-4c19-92b5-f5860b3af647 None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:06.803934 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1f291-7418-4c19-92b5-f5860b3af647 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:06.807559 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 107/221] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:06 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:06.818133 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1f291-7418-4c19-92b5-f5860b3af647 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:06.868176 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618134e-44e9-457f-a1b0-4c24b046e7a2 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:06.927068 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda832a-e617-40af-8a81-987a7eb1859d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:06.927914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda832a-e617-40af-8a81-987a7eb1859d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:06.928221 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda832a-e617-40af-8a81-987a7eb1859d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:06.928493 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda832a-e617-40af-8a81-987a7eb1859d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:06.948150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda832a-e617-40af-8a81-987a7eb1859d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:06.955306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-1eda832a-e617-40af-8a81-987a7eb1859d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:06.955306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:06.955306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:06.955306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:06.955306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:06.955306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:06.955306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:06.955306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:06.955306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:06.955306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:06.955306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:06.955306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:06.955306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:06.955306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:06.955306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:06.955306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:06.957153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:06.957153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:06.957153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:06.957153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:06.957153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:06.957153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:06.957153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:06.957153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:06.957153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:06.957153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:06.957153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:06.957153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:06.957153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:06.957153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:06.957153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:06.957153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:06.959359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:06.959359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:06.959359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:06.959359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:06.959359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:06.959359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:06.959359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:06.959359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:06.959359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:06.959359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:06.959359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:06.959359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:06.959359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 115/222] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:06.964754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda832a-e617-40af-8a81-987a7eb1859d None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:07.040788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:07.081499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1f291-7418-4c19-92b5-f5860b3af647 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:07.114545 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:07.145580 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945e338-a476-4652-a7dd-235417808fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:07.146226 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945e338-a476-4652-a7dd-235417808fec None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:07.146495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945e338-a476-4652-a7dd-235417808fec None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:07.146779 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945e338-a476-4652-a7dd-235417808fec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:07.147507 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945e338-a476-4652-a7dd-235417808fec None admin] RBAC: Adding query filter params (role.id=4eebd8098ab44bfc84cd3de29a679bdf, scope.project.id=b994c620a91d441aa453e5be1b958864, user.id=5b475a64338649da996a33f42448bb5b) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:07.147807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945e338-a476-4652-a7dd-235417808fec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:07.149334 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945e338-a476-4652-a7dd-235417808fec None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:07.157746 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 108/223] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:06 2022] GET /identity/v3/role_assignments?user.id=5b475a64338649da996a33f42448bb5b&scope.project.id=b994c620a91d441aa453e5be1b958864&role.id=4eebd8098ab44bfc84cd3de29a679bdf => generated 259 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:07.191730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda832a-e617-40af-8a81-987a7eb1859d None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:07.243135 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa203b65-b370-410d-8ca4-ab169aa112c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:07.243135 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa203b65-b370-410d-8ca4-ab169aa112c1 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:07.243135 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa203b65-b370-410d-8ca4-ab169aa112c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:07.243609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa203b65-b370-410d-8ca4-ab169aa112c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:07.244219 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa203b65-b370-410d-8ca4-ab169aa112c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:07.244378 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa203b65-b370-410d-8ca4-ab169aa112c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:07.245915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa203b65-b370-410d-8ca4-ab169aa112c1 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:07.257284 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 116/224] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:07.265300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945e338-a476-4652-a7dd-235417808fec None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:07.463937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945e338-a476-4652-a7dd-235417808fec None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:07.505195 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883a84c-cd6d-41af-a02e-9d8855e57810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:07.505783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883a84c-cd6d-41af-a02e-9d8855e57810 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:07.506059 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883a84c-cd6d-41af-a02e-9d8855e57810 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:07.506334 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883a84c-cd6d-41af-a02e-9d8855e57810 None admin] PATH_INFO: `/v3/users/438d211883f446219e02376a2f35daa2` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:07.507629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883a84c-cd6d-41af-a02e-9d8855e57810 None admin] RBAC: Authorizing `identity:get_user(user_id=438d211883f446219e02376a2f35daa2)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:07.509206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883a84c-cd6d-41af-a02e-9d8855e57810 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:07.511234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 109/225] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:07 2022] GET /identity/v3/users/438d211883f446219e02376a2f35daa2 => generated 283 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:07.519894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa203b65-b370-410d-8ca4-ab169aa112c1 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:07.544160 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3179bb-aaf2-4d29-b858-7ce4e99ba221 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:07.544977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3179bb-aaf2-4d29-b858-7ce4e99ba221 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:07.544977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3179bb-aaf2-4d29-b858-7ce4e99ba221 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:07.546359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 110/226] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:30:07.552984 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c197d-bd0f-46b3-9f1a-22d031f0c22c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:07.553366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c197d-bd0f-46b3-9f1a-22d031f0c22c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:07.553543 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c197d-bd0f-46b3-9f1a-22d031f0c22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:07.594557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2f8c197d-bd0f-46b3-9f1a-22d031f0c22c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:07.595907 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2f8c197d-bd0f-46b3-9f1a-22d031f0c22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:07.668761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:07.709667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:07.712621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa203b65-b370-410d-8ca4-ab169aa112c1 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:07.744682 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 111/227] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:30:07.765780 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1f102-7449-42c3-98c8-ddb32d64f210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:07.766447 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1f102-7449-42c3-98c8-ddb32d64f210 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:07.766722 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1f102-7449-42c3-98c8-ddb32d64f210 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:07.767000 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1f102-7449-42c3-98c8-ddb32d64f210 None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:07.768251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1f102-7449-42c3-98c8-ddb32d64f210 None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:07.769708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1f102-7449-42c3-98c8-ddb32d64f210 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:07.771698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 117/228] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:07 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:07.780525 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c197d-bd0f-46b3-9f1a-22d031f0c22c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:07.995675 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c197d-bd0f-46b3-9f1a-22d031f0c22c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:08.043642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eda2e3-02e6-4242-95e4-9fb28bac5ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:08.044262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eda2e3-02e6-4242-95e4-9fb28bac5ae7 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:08.044537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eda2e3-02e6-4242-95e4-9fb28bac5ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:08.044810 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eda2e3-02e6-4242-95e4-9fb28bac5ae7 None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864/users/438d211883f446219e02376a2f35daa2/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:08.046749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eda2e3-02e6-4242-95e4-9fb28bac5ae7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b994c620a91d441aa453e5be1b958864, user_id=438d211883f446219e02376a2f35daa2, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:08.048642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eda2e3-02e6-4242-95e4-9fb28bac5ae7 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:08.071503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 112/229] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:30:07 2022] PUT /identity/v3/projects/b994c620a91d441aa453e5be1b958864/users/438d211883f446219e02376a2f35daa2/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:30:08.327305 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1f102-7449-42c3-98c8-ddb32d64f210 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:08.456591 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1f102-7449-42c3-98c8-ddb32d64f210 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:08.502404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0470c-7035-457c-9103-f264ea6af400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:08.502404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0470c-7035-457c-9103-f264ea6af400 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:08.502404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0470c-7035-457c-9103-f264ea6af400 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:08.502404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0470c-7035-457c-9103-f264ea6af400 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:08.529528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0470c-7035-457c-9103-f264ea6af400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:08.541014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-44a0470c-7035-457c-9103-f264ea6af400 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:08.541014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:08.541014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:08.541014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:08.541014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:08.541014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:08.541014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:08.541014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:08.541014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:08.541014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:08.541014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:08.541014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:08.541014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:08.541014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:08.541014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:08.541014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:08.542738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:08.542738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:08.542738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:08.542738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:08.542738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:08.542738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:08.542738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:08.542738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:08.542738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:08.542738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:08.542738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:08.542738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:08.542738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:08.542738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:08.542738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:08.542738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:08.548309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:08.548309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:08.548309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:08.548309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:08.548309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:08.548309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:08.548309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:08.548309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:08.548309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:08.548309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:08.548309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:08.548309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:08.551461 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 118/230] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:08.560530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eda2e3-02e6-4242-95e4-9fb28bac5ae7 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:08.618817 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:08.706143 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eda2e3-02e6-4242-95e4-9fb28bac5ae7 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:08.748588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7526a4-37fb-49f8-8ccf-e6d7f6df6e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:08.749596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7526a4-37fb-49f8-8ccf-e6d7f6df6e68 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:08.750071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7526a4-37fb-49f8-8ccf-e6d7f6df6e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:08.750636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7526a4-37fb-49f8-8ccf-e6d7f6df6e68 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:08.751553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7526a4-37fb-49f8-8ccf-e6d7f6df6e68 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:08.752112 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7526a4-37fb-49f8-8ccf-e6d7f6df6e68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:08.753757 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7526a4-37fb-49f8-8ccf-e6d7f6df6e68 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:08.763686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 113/231] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:08.771939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0470c-7035-457c-9103-f264ea6af400 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:08.825659 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:08.929916 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0470c-7035-457c-9103-f264ea6af400 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:08.992227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3addd-8c5c-4e8b-82ff-1e003500d300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:08.992889 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3addd-8c5c-4e8b-82ff-1e003500d300 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:08.993165 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3addd-8c5c-4e8b-82ff-1e003500d300 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:08.993437 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3addd-8c5c-4e8b-82ff-1e003500d300 None admin] PATH_INFO: `/v3/users/438d211883f446219e02376a2f35daa2` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:09.001114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3addd-8c5c-4e8b-82ff-1e003500d300 None admin] RBAC: Authorizing `identity:get_user(user_id=438d211883f446219e02376a2f35daa2)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:09.003223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3addd-8c5c-4e8b-82ff-1e003500d300 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:09.005413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 119/232] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:08 2022] GET /identity/v3/users/438d211883f446219e02376a2f35daa2 => generated 283 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:09.014544 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7526a4-37fb-49f8-8ccf-e6d7f6df6e68 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:09.121961 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:09.135291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7526a4-37fb-49f8-8ccf-e6d7f6df6e68 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:09.183744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cc9cd-1d63-4cdc-a0ca-3e34e644d1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:09.184362 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cc9cd-1d63-4cdc-a0ca-3e34e644d1ff None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:09.184636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cc9cd-1d63-4cdc-a0ca-3e34e644d1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:09.184907 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cc9cd-1d63-4cdc-a0ca-3e34e644d1ff None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:09.186148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cc9cd-1d63-4cdc-a0ca-3e34e644d1ff None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:09.187694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cc9cd-1d63-4cdc-a0ca-3e34e644d1ff None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:09.189568 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 114/233] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:09 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:09.198676 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3addd-8c5c-4e8b-82ff-1e003500d300 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:09.397016 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:09.409640 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3addd-8c5c-4e8b-82ff-1e003500d300 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:09.483900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b1655-c510-449e-8474-af4a38620fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:09.485152 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b1655-c510-449e-8474-af4a38620fdd None admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:09.485621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b1655-c510-449e-8474-af4a38620fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:09.486099 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b1655-c510-449e-8474-af4a38620fdd None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864/users/438d211883f446219e02376a2f35daa2/roles/aedf4e1e5c7848099d7bd183778f5896` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:09.490205 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b1655-c510-449e-8474-af4a38620fdd None admin] RBAC: Authorizing `identity:create_grant(project_id=b994c620a91d441aa453e5be1b958864, user_id=438d211883f446219e02376a2f35daa2, role_id=aedf4e1e5c7848099d7bd183778f5896)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:09.493040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b1655-c510-449e-8474-af4a38620fdd None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:09.519231 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 120/234] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:30:09 2022] PUT /identity/v3/projects/b994c620a91d441aa453e5be1b958864/users/438d211883f446219e02376a2f35daa2/roles/aedf4e1e5c7848099d7bd183778f5896 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:30:10.453848 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0a05-d2a8-4f08-8a8d-d4c646e02a37 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:10.454384 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0a05-d2a8-4f08-8a8d-d4c646e02a37 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:10.454613 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0a05-d2a8-4f08-8a8d-d4c646e02a37 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:10.457373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30208f-48c2-4c7a-b9be-e091b9801cf4 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:10.457828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30208f-48c2-4c7a-b9be-e091b9801cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:10.457904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30208f-48c2-4c7a-b9be-e091b9801cf4 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:10.460832 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 115/235] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:10.461611 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 121/236] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:10.466468 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec2517-557d-425e-b196-9d35ae8bf2cb None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:10.466746 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec2517-557d-425e-b196-9d35ae8bf2cb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:10.467013 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec2517-557d-425e-b196-9d35ae8bf2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:10.472176 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd5853-b589-4245-a6c2-f5b5525fc0a1 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:10.472493 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd5853-b589-4245-a6c2-f5b5525fc0a1 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:10.472720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd5853-b589-4245-a6c2-f5b5525fc0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:10.492812 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-50ec2517-557d-425e-b196-9d35ae8bf2cb None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:10.494013 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-50ec2517-557d-425e-b196-9d35ae8bf2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:10.495199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-60bd5853-b589-4245-a6c2-f5b5525fc0a1 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:10.496441 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-60bd5853-b589-4245-a6c2-f5b5525fc0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:10.573319 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 122/237] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:30:10.575068 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4104e-46b9-471a-89f0-100aefb4b289 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:10.575523 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4104e-46b9-471a-89f0-100aefb4b289 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:10.575986 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4104e-46b9-471a-89f0-100aefb4b289 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:10.579261 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 123/238] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:10.586398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeebd7c-98ba-4a28-ae88-acfa3363ca21 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:10.586681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeebd7c-98ba-4a28-ae88-acfa3363ca21 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:10.587017 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeebd7c-98ba-4a28-ae88-acfa3363ca21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:10.609525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-8aeebd7c-98ba-4a28-ae88-acfa3363ca21 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:10.611126 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-8aeebd7c-98ba-4a28-ae88-acfa3363ca21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:10.616149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 116/239] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:10.714515 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 124/240] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:10.869346 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578204c0-471f-4f0f-96e1-df3585b43d79 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:10.870085 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578204c0-471f-4f0f-96e1-df3585b43d79 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:10.870688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578204c0-471f-4f0f-96e1-df3585b43d79 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:10.872327 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 117/241] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:10.878684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac34c6-7504-48f9-a0b7-b005bccfb3a9 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:10.879276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac34c6-7504-48f9-a0b7-b005bccfb3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:10.879759 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac34c6-7504-48f9-a0b7-b005bccfb3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:10.899050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578204c0-471f-4f0f-96e1-df3585b43d79 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:10.917074 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-b5ac34c6-7504-48f9-a0b7-b005bccfb3a9 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:10.919356 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b5ac34c6-7504-48f9-a0b7-b005bccfb3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:10.947512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:11.059182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:11.068485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 125/242] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:30:11.068485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac34c6-7504-48f9-a0b7-b005bccfb3a9 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:11.085818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578204c0-471f-4f0f-96e1-df3585b43d79 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:11.131336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1945f4-f141-4bc9-84e8-f7802d644088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:11.132012 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1945f4-f141-4bc9-84e8-f7802d644088 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:11.132291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1945f4-f141-4bc9-84e8-f7802d644088 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:11.132559 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1945f4-f141-4bc9-84e8-f7802d644088 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:11.133800 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1945f4-f141-4bc9-84e8-f7802d644088 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:11.135378 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1945f4-f141-4bc9-84e8-f7802d644088 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:11.141594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 118/243] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:11.141594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1945f4-f141-4bc9-84e8-f7802d644088 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:11.188292 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac34c6-7504-48f9-a0b7-b005bccfb3a9 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:11.229672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349ab95-e851-46b8-a58a-8eea7926f322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:11.230683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349ab95-e851-46b8-a58a-8eea7926f322 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:11.231154 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349ab95-e851-46b8-a58a-8eea7926f322 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:11.231642 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349ab95-e851-46b8-a58a-8eea7926f322 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:11.243507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349ab95-e851-46b8-a58a-8eea7926f322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:11.248773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-f349ab95-e851-46b8-a58a-8eea7926f322 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:11.248773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:11.248773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:11.248773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:11.248773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:11.248773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:11.248773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:11.248773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:11.248773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:11.248773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:11.248773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:11.248773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:11.248773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:11.248773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:11.248773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:11.248773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:11.249708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:11.249708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:11.249708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:11.249708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:11.249708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:11.249708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:11.249708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:11.249708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:11.249708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:11.249708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:11.249708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:11.249708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:11.249708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:11.249708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:11.249708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:11.249708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:11.250540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:11.250540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:11.250540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:11.250540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:11.250540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:11.250540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:11.250540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:11.250540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:11.250540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:11.250540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:11.250540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:11.250540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:11.253248 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 126/244] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:11.254409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349ab95-e851-46b8-a58a-8eea7926f322 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:11.254524 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1945f4-f141-4bc9-84e8-f7802d644088 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:11.264517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:11.295611 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda8a88-ad1d-4173-bb90-75098993c6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:11.296342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda8a88-ad1d-4173-bb90-75098993c6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:11.296617 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda8a88-ad1d-4173-bb90-75098993c6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:11.296932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda8a88-ad1d-4173-bb90-75098993c6c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:11.309122 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda8a88-ad1d-4173-bb90-75098993c6c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:11.314815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-edda8a88-ad1d-4173-bb90-75098993c6c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:30:11.314815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:11.314815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:11.314815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:11.314815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:11.314815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:11.314815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:11.314815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:11.314815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:11.314815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:11.314815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:11.314815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:11.314815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:11.314815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:11.314815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:11.314815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:11.315733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:11.315733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:11.315733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:11.315733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:11.315733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:11.315733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:11.315733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:11.315733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:11.315733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:11.315733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:11.315733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:11.315733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:11.315733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:11.315733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:11.315733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:11.315733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:11.316557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:11.316557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:11.316557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:11.316557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:11.316557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:11.316557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:11.316557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:11.316557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:11.316557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:11.316557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:11.316557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:30:11.316557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:11.316557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 119/245] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:11.317215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda8a88-ad1d-4173-bb90-75098993c6c1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:11.352383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:11.373870 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349ab95-e851-46b8-a58a-8eea7926f322 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:11.410858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8c08e-978a-4ba0-b6fa-48e160b41925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:11.411842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8c08e-978a-4ba0-b6fa-48e160b41925 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:11.412466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8c08e-978a-4ba0-b6fa-48e160b41925 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:11.412929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8c08e-978a-4ba0-b6fa-48e160b41925 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:11.413801 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8c08e-978a-4ba0-b6fa-48e160b41925 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:11.415569 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8c08e-978a-4ba0-b6fa-48e160b41925 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:11.434939 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda8a88-ad1d-4173-bb90-75098993c6c1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:11.442383 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 127/246] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:30:11 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:11.443602 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8c08e-978a-4ba0-b6fa-48e160b41925 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:11.505447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b53f5-4c8a-4e75-81e6-f636b83b9e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:11.506187 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b53f5-4c8a-4e75-81e6-f636b83b9e99 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:11.506464 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b53f5-4c8a-4e75-81e6-f636b83b9e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:11.506915 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b53f5-4c8a-4e75-81e6-f636b83b9e99 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:11.507912 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b53f5-4c8a-4e75-81e6-f636b83b9e99 None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:11.508292 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b53f5-4c8a-4e75-81e6-f636b83b9e99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:11.510065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b53f5-4c8a-4e75-81e6-f636b83b9e99 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:11.521708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 120/247] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:11.528290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b53f5-4c8a-4e75-81e6-f636b83b9e99 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:11.572608 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:11.632121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:11.651873 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8c08e-978a-4ba0-b6fa-48e160b41925 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:11.656647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b53f5-4c8a-4e75-81e6-f636b83b9e99 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:11.702763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551fc9a-4569-4c91-ba0b-5f26b21f4c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:11.704632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551fc9a-4569-4c91-ba0b-5f26b21f4c8c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:11.706939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551fc9a-4569-4c91-ba0b-5f26b21f4c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:11.707697 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551fc9a-4569-4c91-ba0b-5f26b21f4c8c None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:11.709664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5664ff-e5c6-44f9-ac41-3df5b9c32220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:11.710075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551fc9a-4569-4c91-ba0b-5f26b21f4c8c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:11.710520 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551fc9a-4569-4c91-ba0b-5f26b21f4c8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:11.710712 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5664ff-e5c6-44f9-ac41-3df5b9c32220 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:11.711195 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5664ff-e5c6-44f9-ac41-3df5b9c32220 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:11.711541 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5664ff-e5c6-44f9-ac41-3df5b9c32220 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:11.712580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551fc9a-4569-4c91-ba0b-5f26b21f4c8c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:11.728063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5664ff-e5c6-44f9-ac41-3df5b9c32220 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:11.728541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 128/248] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:11.729776 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551fc9a-4569-4c91-ba0b-5f26b21f4c8c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:11.734553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-fa5664ff-e5c6-44f9-ac41-3df5b9c32220 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:11.734553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:11.734553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:11.734553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:11.734553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:11.734553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:11.734553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:11.734553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:11.734553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:11.734553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:11.734553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:11.734553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:11.734553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:11.734553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:11.734553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:11.734553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:11.735819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:11.735819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:11.735819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:11.735819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:11.735819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:11.735819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:11.735819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:11.735819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:11.735819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:11.735819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:11.735819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:11.735819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:11.735819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:11.735819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:11.735819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:11.735819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:11.736645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:11.736645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:11.736645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:11.736645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:11.736645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:11.736645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:11.736645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:11.736645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:11.736645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:11.736645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:11.736645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:11.736645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:11.736645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 121/249] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:11.741077 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5664ff-e5c6-44f9-ac41-3df5b9c32220 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:11.866814 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551fc9a-4569-4c91-ba0b-5f26b21f4c8c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:11.895384 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:11.912879 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c9933-ff03-44d0-ae8b-03b0e18b39c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:11.913547 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c9933-ff03-44d0-ae8b-03b0e18b39c7 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:11.913849 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c9933-ff03-44d0-ae8b-03b0e18b39c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:11.914263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c9933-ff03-44d0-ae8b-03b0e18b39c7 None admin] PATH_INFO: `/v3/users/5b475a64338649da996a33f42448bb5b` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:11.915685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c9933-ff03-44d0-ae8b-03b0e18b39c7 None admin] RBAC: Authorizing `identity:get_user(user_id=5b475a64338649da996a33f42448bb5b)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:11.918094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c9933-ff03-44d0-ae8b-03b0e18b39c7 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:11.919964 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 129/250] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:11 2022] GET /identity/v3/users/5b475a64338649da996a33f42448bb5b => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:11.922506 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c9933-ff03-44d0-ae8b-03b0e18b39c7 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:11.938962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5664ff-e5c6-44f9-ac41-3df5b9c32220 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:11.978131 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd1839-ac9d-414b-a9ce-049a937c4a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:11.978790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd1839-ac9d-414b-a9ce-049a937c4a21 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:11.979077 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd1839-ac9d-414b-a9ce-049a937c4a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:11.979344 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd1839-ac9d-414b-a9ce-049a937c4a21 None admin] PATH_INFO: `/v3/users/de0075f3771b49cdb643ab2fbbc2dfd1` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:11.991696 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:11.997233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd1839-ac9d-414b-a9ce-049a937c4a21 None admin] RBAC: Authorizing `identity:get_user(user_id=de0075f3771b49cdb643ab2fbbc2dfd1)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:11.999115 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd1839-ac9d-414b-a9ce-049a937c4a21 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:12.001065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 122/251] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:11 2022] GET /identity/v3/users/de0075f3771b49cdb643ab2fbbc2dfd1 => generated 297 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:12.001997 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd1839-ac9d-414b-a9ce-049a937c4a21 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:12.140420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c9933-ff03-44d0-ae8b-03b0e18b39c7 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:12.146635 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd1839-ac9d-414b-a9ce-049a937c4a21 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:12.187252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98545a58-9620-4103-9801-06eb8f3e4099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:12.187992 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98545a58-9620-4103-9801-06eb8f3e4099 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:12.188389 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98545a58-9620-4103-9801-06eb8f3e4099 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:12.188661 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98545a58-9620-4103-9801-06eb8f3e4099 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:12.189356 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98545a58-9620-4103-9801-06eb8f3e4099 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:12.189665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98545a58-9620-4103-9801-06eb8f3e4099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:12.191151 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98545a58-9620-4103-9801-06eb8f3e4099 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:12.191480 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5906d-4ba7-45af-b22c-eecf2d54118c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:12.192343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5906d-4ba7-45af-b22c-eecf2d54118c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:12.192765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5906d-4ba7-45af-b22c-eecf2d54118c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:12.193204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5906d-4ba7-45af-b22c-eecf2d54118c None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:12.194550 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5906d-4ba7-45af-b22c-eecf2d54118c None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:12.196232 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5906d-4ba7-45af-b22c-eecf2d54118c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:12.198097 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 123/252] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:12 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:12.204096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 130/253] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:12.204096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af9ded-acec-42e0-8cb6-89c9f75b6589 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:12.204096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af9ded-acec-42e0-8cb6-89c9f75b6589 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:12.204096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af9ded-acec-42e0-8cb6-89c9f75b6589 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:12.204972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 131/254] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:12.205335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5906d-4ba7-45af-b22c-eecf2d54118c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:12.205714 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af9ded-acec-42e0-8cb6-89c9f75b6589 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:12.293430 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:12.302911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:12.395404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5906d-4ba7-45af-b22c-eecf2d54118c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:12.405954 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af9ded-acec-42e0-8cb6-89c9f75b6589 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:12.474265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695eea7a-3a3a-4cbe-b714-85a5fd4a1e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:12.475185 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695eea7a-3a3a-4cbe-b714-85a5fd4a1e6f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:12.475607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695eea7a-3a3a-4cbe-b714-85a5fd4a1e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:12.476047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695eea7a-3a3a-4cbe-b714-85a5fd4a1e6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:12.477394 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa49f41-cba8-47df-aa80-5ce5bdeebe6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:12.478548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa49f41-cba8-47df-aa80-5ce5bdeebe6a None admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:12.479025 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa49f41-cba8-47df-aa80-5ce5bdeebe6a None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:12.479493 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa49f41-cba8-47df-aa80-5ce5bdeebe6a None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864/users/5b475a64338649da996a33f42448bb5b/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:12.482918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa49f41-cba8-47df-aa80-5ce5bdeebe6a None admin] RBAC: Authorizing `identity:create_grant(project_id=b994c620a91d441aa453e5be1b958864, user_id=5b475a64338649da996a33f42448bb5b, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:12.485789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa49f41-cba8-47df-aa80-5ce5bdeebe6a None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:12.536049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 132/255] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:30:12 2022] PUT /identity/v3/projects/b994c620a91d441aa453e5be1b958864/users/5b475a64338649da996a33f42448bb5b/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:30:12.536049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabccdd3-e785-4841-a213-41a5b1899ba9 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:12.536049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabccdd3-e785-4841-a213-41a5b1899ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:12.536416 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabccdd3-e785-4841-a213-41a5b1899ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:12.576197 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-eabccdd3-e785-4841-a213-41a5b1899ba9 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:12.584040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-eabccdd3-e785-4841-a213-41a5b1899ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:12.642987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695eea7a-3a3a-4cbe-b714-85a5fd4a1e6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:12.646655 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695eea7a-3a3a-4cbe-b714-85a5fd4a1e6f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:12.698031 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 133/256] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:12.699176 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabccdd3-e785-4841-a213-41a5b1899ba9 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:12.808220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:12.930167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:12.940594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabccdd3-e785-4841-a213-41a5b1899ba9 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:13.011776 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 124/257] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 813 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:13.019461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695eea7a-3a3a-4cbe-b714-85a5fd4a1e6f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:13.032512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2646295-1b72-48f0-b327-ecea3fb31a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:13.033616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2646295-1b72-48f0-b327-ecea3fb31a18 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:13.034118 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2646295-1b72-48f0-b327-ecea3fb31a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:13.034584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2646295-1b72-48f0-b327-ecea3fb31a18 None admin] PATH_INFO: `/v3/users/438d211883f446219e02376a2f35daa2` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:13.036204 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2646295-1b72-48f0-b327-ecea3fb31a18 None admin] RBAC: Authorizing `identity:get_user(user_id=438d211883f446219e02376a2f35daa2)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:13.040462 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2646295-1b72-48f0-b327-ecea3fb31a18 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:13.044361 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 134/258] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:12 2022] GET /identity/v3/users/438d211883f446219e02376a2f35daa2 => generated 283 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:13.053466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2646295-1b72-48f0-b327-ecea3fb31a18 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:13.310637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2646295-1b72-48f0-b327-ecea3fb31a18 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:13.330409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:13.343648 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695eea7a-3a3a-4cbe-b714-85a5fd4a1e6f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:13.384160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67965cbe-2b0d-4332-b3f9-1652fcda25a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:13.385314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67965cbe-2b0d-4332-b3f9-1652fcda25a4 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:13.386802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67965cbe-2b0d-4332-b3f9-1652fcda25a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:13.387306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67965cbe-2b0d-4332-b3f9-1652fcda25a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:13.462519 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdd174-6330-4b52-99cb-26f9b40af5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:13.465071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdd174-6330-4b52-99cb-26f9b40af5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:13.467844 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdd174-6330-4b52-99cb-26f9b40af5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:13.469134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdd174-6330-4b52-99cb-26f9b40af5a4 None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:13.470679 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fdd174-6330-4b52-99cb-26f9b40af5a4 None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:13.473959 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fdd174-6330-4b52-99cb-26f9b40af5a4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:13.476784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 125/259] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:13 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:13.479649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdd174-6330-4b52-99cb-26f9b40af5a4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:13.528969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67965cbe-2b0d-4332-b3f9-1652fcda25a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:13.530813 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67965cbe-2b0d-4332-b3f9-1652fcda25a4 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:13.550482 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:13.713377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdd174-6330-4b52-99cb-26f9b40af5a4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:13.765229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 135/260] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:13.766163 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67965cbe-2b0d-4332-b3f9-1652fcda25a4 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:13.792272 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10dc27-2834-4fb1-b6ac-683717b57679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:13.792933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10dc27-2834-4fb1-b6ac-683717b57679 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:13.793211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10dc27-2834-4fb1-b6ac-683717b57679 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:13.793481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10dc27-2834-4fb1-b6ac-683717b57679 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:13.804384 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:13.823157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10dc27-2834-4fb1-b6ac-683717b57679 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:13.828104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-bc10dc27-2834-4fb1-b6ac-683717b57679 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:13.828104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:13.828104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:13.828104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:13.828104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:13.828104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:13.828104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:13.828104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:13.828104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:13.828104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:13.828104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:13.828104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:13.828104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:13.828104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:13.828104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:13.828104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:13.829060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:13.829060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:13.829060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:13.829060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:13.829060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:13.829060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:13.829060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:13.829060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:13.829060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:13.829060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:13.829060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:13.829060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:13.829060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:13.829060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:13.829060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:13.829060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:13.829838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:13.829838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:13.829838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:13.829838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:13.829838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:13.829838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:13.829838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:13.829838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:13.829838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:13.829838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:13.829838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:13.829838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:13.836860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 126/261] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:13.837777 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10dc27-2834-4fb1-b6ac-683717b57679 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:14.018324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67965cbe-2b0d-4332-b3f9-1652fcda25a4 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:14.072323 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:14.086805 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10dc27-2834-4fb1-b6ac-683717b57679 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:14.106233 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3bd1e-3e1c-4dbb-a88b-765dbdbf25d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:14.107028 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3bd1e-3e1c-4dbb-a88b-765dbdbf25d1 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.107310 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3bd1e-3e1c-4dbb-a88b-765dbdbf25d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.107577 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3bd1e-3e1c-4dbb-a88b-765dbdbf25d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:14.108375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3bd1e-3e1c-4dbb-a88b-765dbdbf25d1 None admin] RBAC: Adding query filter params (role.id=89f1b9db96764371860f845c00419e7a, scope.project.id=b994c620a91d441aa453e5be1b958864, user.id=de0075f3771b49cdb643ab2fbbc2dfd1) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:14.108683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3bd1e-3e1c-4dbb-a88b-765dbdbf25d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:14.114806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3bd1e-3e1c-4dbb-a88b-765dbdbf25d1 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:14.126129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 136/262] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:13 2022] GET /identity/v3/role_assignments?user.id=de0075f3771b49cdb643ab2fbbc2dfd1&scope.project.id=b994c620a91d441aa453e5be1b958864&role.id=89f1b9db96764371860f845c00419e7a => generated 259 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:14.127049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3bd1e-3e1c-4dbb-a88b-765dbdbf25d1 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:14.163197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7d0f5-d230-4b6a-a201-c9f787ec03b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:14.164293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7d0f5-d230-4b6a-a201-c9f787ec03b5 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.165976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7d0f5-d230-4b6a-a201-c9f787ec03b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.166523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7d0f5-d230-4b6a-a201-c9f787ec03b5 None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:14.168169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7d0f5-d230-4b6a-a201-c9f787ec03b5 None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:14.171919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7d0f5-d230-4b6a-a201-c9f787ec03b5 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:14.183742 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 127/263] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:13 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:14.192234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cea75e-1907-42dd-8679-d2eb1ae1c5d6 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.192858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cea75e-1907-42dd-8679-d2eb1ae1c5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.193597 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cea75e-1907-42dd-8679-d2eb1ae1c5d6 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:14.195605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 128/264] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:14 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:14.196785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cea75e-1907-42dd-8679-d2eb1ae1c5d6 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:14.320663 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:14.375714 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3bd1e-3e1c-4dbb-a88b-765dbdbf25d1 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:14.386763 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cea75e-1907-42dd-8679-d2eb1ae1c5d6 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:14.428326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbe61f-4c61-4db3-885b-1ca83c6e91d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:14.429366 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbe61f-4c61-4db3-885b-1ca83c6e91d2 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.429844 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbe61f-4c61-4db3-885b-1ca83c6e91d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.430309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbe61f-4c61-4db3-885b-1ca83c6e91d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:14.431199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbe61f-4c61-4db3-885b-1ca83c6e91d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:14.431745 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbe61f-4c61-4db3-885b-1ca83c6e91d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:14.433435 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbe61f-4c61-4db3-885b-1ca83c6e91d2 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:14.443686 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 137/265] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:14.445719 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6d469-2923-42b1-a418-80d3fb246f06 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.446215 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6d469-2923-42b1-a418-80d3fb246f06 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.446709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6d469-2923-42b1-a418-80d3fb246f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:14.447467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ac830-705b-4f4b-959e-0675d1dc015c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:14.448594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ac830-705b-4f4b-959e-0675d1dc015c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.449120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ac830-705b-4f4b-959e-0675d1dc015c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.449551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ac830-705b-4f4b-959e-0675d1dc015c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:14.450830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ac830-705b-4f4b-959e-0675d1dc015c None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=b994c620a91d441aa453e5be1b958864, user.id=438d211883f446219e02376a2f35daa2) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:14.451325 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ac830-705b-4f4b-959e-0675d1dc015c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:14.454421 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ac830-705b-4f4b-959e-0675d1dc015c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:14.469073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 129/266] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:14 2022] GET /identity/v3/role_assignments?user.id=438d211883f446219e02376a2f35daa2&scope.project.id=b994c620a91d441aa453e5be1b958864&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:14.476179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef71e3c-c583-4722-b1de-1fb39c70c4e4 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.476179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef71e3c-c583-4722-b1de-1fb39c70c4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.476179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef71e3c-c583-4722-b1de-1fb39c70c4e4 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:14.476179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 130/267] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:14.476747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef71e3c-c583-4722-b1de-1fb39c70c4e4 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:14.481530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a3c6d469-2923-42b1-a418-80d3fb246f06 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:14.483086 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a3c6d469-2923-42b1-a418-80d3fb246f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:14.554505 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:14.691332 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 138/268] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:14.700025 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9172b-f324-4cce-9389-65c8dde6303f None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.700025 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9172b-f324-4cce-9389-65c8dde6303f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.700025 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9172b-f324-4cce-9389-65c8dde6303f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:14.761588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-1be9172b-f324-4cce-9389-65c8dde6303f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:14.763644 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-1be9172b-f324-4cce-9389-65c8dde6303f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:14.786176 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef71e3c-c583-4722-b1de-1fb39c70c4e4 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:14.858318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76658c6c-fcf0-4aa6-a941-7df85fc70648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:14.858978 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76658c6c-fcf0-4aa6-a941-7df85fc70648 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.859417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76658c6c-fcf0-4aa6-a941-7df85fc70648 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.862593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76658c6c-fcf0-4aa6-a941-7df85fc70648 None admin] PATH_INFO: `/v3/users/438d211883f446219e02376a2f35daa2` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:14.865385 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76658c6c-fcf0-4aa6-a941-7df85fc70648 None admin] RBAC: Authorizing `identity:get_user(user_id=438d211883f446219e02376a2f35daa2)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:14.867171 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76658c6c-fcf0-4aa6-a941-7df85fc70648 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:14.869126 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 131/269] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:14 2022] GET /identity/v3/users/438d211883f446219e02376a2f35daa2 => generated 283 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:14.875717 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76658c6c-fcf0-4aa6-a941-7df85fc70648 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:14.917307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:14.933439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 139/270] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:14.996399 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:14.999420 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76658c6c-fcf0-4aa6-a941-7df85fc70648 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:15.041065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca92b08-7e13-4f16-82a9-53bfcf877763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:15.041715 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca92b08-7e13-4f16-82a9-53bfcf877763 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:15.041995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca92b08-7e13-4f16-82a9-53bfcf877763 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:15.042262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca92b08-7e13-4f16-82a9-53bfcf877763 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:15.118256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca92b08-7e13-4f16-82a9-53bfcf877763 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:15.119831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca92b08-7e13-4f16-82a9-53bfcf877763 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:15.208403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9172b-f324-4cce-9389-65c8dde6303f None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:15.223089 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:15.230685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 132/271] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:15.239567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ec6f0-b377-433f-be62-25650681e812 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:15.239849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ec6f0-b377-433f-be62-25650681e812 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:15.240149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ec6f0-b377-433f-be62-25650681e812 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:15.241725 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 133/272] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:15.242347 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ec6f0-b377-433f-be62-25650681e812 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:15.379889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ec6f0-b377-433f-be62-25650681e812 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:15.415154 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9172b-f324-4cce-9389-65c8dde6303f None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:15.425830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ecf393-80f3-47fa-826d-a1a06b8201b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:15.426486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecf393-80f3-47fa-826d-a1a06b8201b3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:15.426761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecf393-80f3-47fa-826d-a1a06b8201b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:15.427028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecf393-80f3-47fa-826d-a1a06b8201b3 None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:15.428233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ecf393-80f3-47fa-826d-a1a06b8201b3 None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:15.429873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ecf393-80f3-47fa-826d-a1a06b8201b3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:15.431403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 134/273] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:15 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:15.436678 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2498a-ac4a-43ea-84e1-4a3c7905799d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:15.436678 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2498a-ac4a-43ea-84e1-4a3c7905799d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:15.436678 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2498a-ac4a-43ea-84e1-4a3c7905799d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:15.465856 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7613ea31-bdee-4fed-bdfc-8867952af695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:15.466398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-50c2498a-ac4a-43ea-84e1-4a3c7905799d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:15.466504 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613ea31-bdee-4fed-bdfc-8867952af695 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:15.466751 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613ea31-bdee-4fed-bdfc-8867952af695 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:15.467024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613ea31-bdee-4fed-bdfc-8867952af695 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:15.468440 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-50c2498a-ac4a-43ea-84e1-4a3c7905799d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:15.480143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7613ea31-bdee-4fed-bdfc-8867952af695 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:15.486155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-7613ea31-bdee-4fed-bdfc-8867952af695 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:15.486155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:15.486155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:15.486155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:15.486155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:15.486155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:15.486155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:15.486155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:15.486155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:15.486155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:15.486155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:15.486155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:15.486155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:15.486155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:15.486155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:15.486155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:15.487114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:15.487114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:15.487114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:15.487114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:15.487114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:15.487114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:15.487114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:15.487114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:15.487114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:15.487114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:15.487114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:15.487114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:15.487114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:15.487114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:15.487114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:15.487114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:15.487892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:15.487892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:15.487892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:15.487892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:15.487892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:15.487892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:15.487892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:15.487892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:15.487892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:15.487892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:15.487892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:15.487892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:15.487892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 140/274] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:15.487892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7613ea31-bdee-4fed-bdfc-8867952af695 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:15.547541 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:15.561340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 135/275] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:15.562168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2498a-ac4a-43ea-84e1-4a3c7905799d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:15.582051 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:15.670270 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7613ea31-bdee-4fed-bdfc-8867952af695 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:15.745186 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af0765-20a3-478f-9c62-f19b5688fbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:15.746568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af0765-20a3-478f-9c62-f19b5688fbb2 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:15.747556 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af0765-20a3-478f-9c62-f19b5688fbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:15.748779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af0765-20a3-478f-9c62-f19b5688fbb2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:15.752337 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af0765-20a3-478f-9c62-f19b5688fbb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:15.755287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af0765-20a3-478f-9c62-f19b5688fbb2 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:15.758925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 141/276] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:15.760439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2498a-ac4a-43ea-84e1-4a3c7905799d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:15.760955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af0765-20a3-478f-9c62-f19b5688fbb2 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:15.829544 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b918690-a828-4b66-a473-9c4cd4ed3059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:15.830334 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b918690-a828-4b66-a473-9c4cd4ed3059 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:15.830840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b918690-a828-4b66-a473-9c4cd4ed3059 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:15.831484 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b918690-a828-4b66-a473-9c4cd4ed3059 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:15.832808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b918690-a828-4b66-a473-9c4cd4ed3059 None admin] RBAC: Adding query filter params (role.id=aedf4e1e5c7848099d7bd183778f5896, scope.project.id=b994c620a91d441aa453e5be1b958864, user.id=438d211883f446219e02376a2f35daa2) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:15.833324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b918690-a828-4b66-a473-9c4cd4ed3059 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:15.841981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b918690-a828-4b66-a473-9c4cd4ed3059 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:15.850430 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 136/277] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:15 2022] GET /identity/v3/role_assignments?user.id=438d211883f446219e02376a2f35daa2&scope.project.id=b994c620a91d441aa453e5be1b958864&role.id=aedf4e1e5c7848099d7bd183778f5896 => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:15.851294 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b918690-a828-4b66-a473-9c4cd4ed3059 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:15.951741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af0765-20a3-478f-9c62-f19b5688fbb2 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:15.968601 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:15.990781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:16.001803 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c84248-5dab-4821-ab50-3d7847a1c4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:16.002682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c84248-5dab-4821-ab50-3d7847a1c4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.003098 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c84248-5dab-4821-ab50-3d7847a1c4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.003518 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c84248-5dab-4821-ab50-3d7847a1c4f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.004407 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c84248-5dab-4821-ab50-3d7847a1c4f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:16.004851 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c84248-5dab-4821-ab50-3d7847a1c4f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:16.006406 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c84248-5dab-4821-ab50-3d7847a1c4f9 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:16.016763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 142/278] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:16.017970 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c84248-5dab-4821-ab50-3d7847a1c4f9 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:16.063218 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b918690-a828-4b66-a473-9c4cd4ed3059 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:16.129239 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa11001-2f9c-46ec-a7c0-9b88a5a72e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:16.130722 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa11001-2f9c-46ec-a7c0-9b88a5a72e70 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.131170 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa11001-2f9c-46ec-a7c0-9b88a5a72e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.131612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa11001-2f9c-46ec-a7c0-9b88a5a72e70 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.132749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa11001-2f9c-46ec-a7c0-9b88a5a72e70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:16.135233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa11001-2f9c-46ec-a7c0-9b88a5a72e70 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:16.143673 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c84248-5dab-4821-ab50-3d7847a1c4f9 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:16.178086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 137/279] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:30:15 2022] POST /identity/v3/users => generated 293 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:16.178086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa11001-2f9c-46ec-a7c0-9b88a5a72e70 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:16.197231 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a903f-bf67-42c8-84e3-6728ee8fe292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:16.197998 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a903f-bf67-42c8-84e3-6728ee8fe292 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.198338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a903f-bf67-42c8-84e3-6728ee8fe292 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.198665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a903f-bf67-42c8-84e3-6728ee8fe292 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.215317 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a903f-bf67-42c8-84e3-6728ee8fe292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:16.222769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-883a903f-bf67-42c8-84e3-6728ee8fe292 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:16.222769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:16.222769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:16.222769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:16.222769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:16.222769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:16.222769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:16.222769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:16.222769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:16.222769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:16.222769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:16.222769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:16.222769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:16.222769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:16.222769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:16.222769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:16.224419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:16.224419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:16.224419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:16.224419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:16.224419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:16.224419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:16.224419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:16.224419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:16.224419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:16.224419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:16.224419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:16.224419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:16.224419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:16.224419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:16.224419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:16.224419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:16.225944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:16.225944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:16.225944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:16.225944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:16.225944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:16.225944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:16.225944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:16.225944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:16.225944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:16.225944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:16.225944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:16.225944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:16.230736 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 143/280] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:16.231744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a903f-bf67-42c8-84e3-6728ee8fe292 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:16.326718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:16.418188 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa11001-2f9c-46ec-a7c0-9b88a5a72e70 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:16.438817 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a903f-bf67-42c8-84e3-6728ee8fe292 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:16.440886 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:16.476343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d77426-f98a-414e-bd2f-3147a68669d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:16.477352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d77426-f98a-414e-bd2f-3147a68669d5 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.477793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d77426-f98a-414e-bd2f-3147a68669d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.478258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d77426-f98a-414e-bd2f-3147a68669d5 None admin] PATH_INFO: `/v3/users/cda0c9596f0347198afa747d65b1c505` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.491683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb0bdc-a544-4832-a761-c53b4ef138bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:16.492644 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb0bdc-a544-4832-a761-c53b4ef138bb None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.493089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb0bdc-a544-4832-a761-c53b4ef138bb None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.493550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb0bdc-a544-4832-a761-c53b4ef138bb None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.494468 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cb0bdc-a544-4832-a761-c53b4ef138bb None admin] RBAC: Adding query filter params (name=member) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:16.494915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cb0bdc-a544-4832-a761-c53b4ef138bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:16.496555 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cb0bdc-a544-4832-a761-c53b4ef138bb None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:16.501454 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d77426-f98a-414e-bd2f-3147a68669d5 None admin] RBAC: Authorizing `identity:get_user(user_id=cda0c9596f0347198afa747d65b1c505)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:16.504241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d77426-f98a-414e-bd2f-3147a68669d5 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:16.512054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 138/281] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:16 2022] GET /identity/v3/users/cda0c9596f0347198afa747d65b1c505 => generated 293 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:16.512054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621e8a5-5684-4bac-8d86-c46e58abbf0e None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.512054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621e8a5-5684-4bac-8d86-c46e58abbf0e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.512054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621e8a5-5684-4bac-8d86-c46e58abbf0e None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.513491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 139/282] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:16.514470 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621e8a5-5684-4bac-8d86-c46e58abbf0e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:16.514770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 144/283] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:16.521647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb0bdc-a544-4832-a761-c53b4ef138bb None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:16.687176 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621e8a5-5684-4bac-8d86-c46e58abbf0e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:16.704447 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb0bdc-a544-4832-a761-c53b4ef138bb None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:16.717446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:16.753373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0034555-fb24-43ff-b294-4253d6f2d08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:16.754479 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0034555-fb24-43ff-b294-4253d6f2d08f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.754479 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0034555-fb24-43ff-b294-4253d6f2d08f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.754766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0034555-fb24-43ff-b294-4253d6f2d08f None admin] PATH_INFO: `/v3/users/5b475a64338649da996a33f42448bb5b` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.756224 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0034555-fb24-43ff-b294-4253d6f2d08f None admin] RBAC: Authorizing `identity:get_user(user_id=5b475a64338649da996a33f42448bb5b)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:16.757640 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0034555-fb24-43ff-b294-4253d6f2d08f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:16.759679 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 145/284] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:16 2022] GET /identity/v3/users/5b475a64338649da996a33f42448bb5b => generated 297 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:16.761103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec595af5-02b1-4583-b793-523cfcea260e None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.761434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec595af5-02b1-4583-b793-523cfcea260e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.761795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec595af5-02b1-4583-b793-523cfcea260e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.764771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea87fe8-fcc8-47b8-8a66-9aeede1219c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:16.766100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea87fe8-fcc8-47b8-8a66-9aeede1219c1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.766806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea87fe8-fcc8-47b8-8a66-9aeede1219c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.767499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea87fe8-fcc8-47b8-8a66-9aeede1219c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.769466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea87fe8-fcc8-47b8-8a66-9aeede1219c1 None admin] RBAC: Adding query filter params (role.id=4eebd8098ab44bfc84cd3de29a679bdf, user.id=cda0c9596f0347198afa747d65b1c505, scope.system=all) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:16.771526 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea87fe8-fcc8-47b8-8a66-9aeede1219c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:16.774468 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea87fe8-fcc8-47b8-8a66-9aeede1219c1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:16.783246 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-ec595af5-02b1-4583-b793-523cfcea260e None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:16.784673 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-ec595af5-02b1-4583-b793-523cfcea260e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:16.786000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 140/285] 10.222.0.25 () {56 vars in 1308 bytes} [Tue Aug 30 14:30:16 2022] GET /identity/v3/role_assignments?user.id=cda0c9596f0347198afa747d65b1c505&scope.system=all&role.id=4eebd8098ab44bfc84cd3de29a679bdf => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:16.787736 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea87fe8-fcc8-47b8-8a66-9aeede1219c1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:16.871232 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:17.031293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea87fe8-fcc8-47b8-8a66-9aeede1219c1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:17.032832 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 146/286] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:17.103293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2512e-7016-4dcd-b2ad-caaa5b636526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:17.104904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2512e-7016-4dcd-b2ad-caaa5b636526 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:17.105676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2512e-7016-4dcd-b2ad-caaa5b636526 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:17.106405 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2512e-7016-4dcd-b2ad-caaa5b636526 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:17.232136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2512e-7016-4dcd-b2ad-caaa5b636526 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:17.236087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2512e-7016-4dcd-b2ad-caaa5b636526 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:17.340606 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:17.342988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec595af5-02b1-4583-b793-523cfcea260e None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:17.468041 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 141/287] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 674 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:17.469381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2512e-7016-4dcd-b2ad-caaa5b636526 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:17.667088 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec595af5-02b1-4583-b793-523cfcea260e None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:17.729442 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:17.797103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2512e-7016-4dcd-b2ad-caaa5b636526 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:17.941022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb13c7-9db5-44bf-8555-d7b3db43ed77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:17.941600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb13c7-9db5-44bf-8555-d7b3db43ed77 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:17.942028 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb13c7-9db5-44bf-8555-d7b3db43ed77 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:17.942169 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb13c7-9db5-44bf-8555-d7b3db43ed77 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:17.983688 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb13c7-9db5-44bf-8555-d7b3db43ed77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:17.989044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-c3fb13c7-9db5-44bf-8555-d7b3db43ed77 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:30:17.989044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:17.989044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:17.989044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:17.989044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:17.989044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:17.989044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:17.989044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:17.989044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:17.989044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:17.989044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:17.989044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:17.989044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:17.989044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:17.989044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:17.989044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:17.990671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:17.990671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:17.990671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:17.990671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:17.990671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:17.990671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:17.990671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:17.990671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:17.990671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:17.990671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:17.990671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:17.990671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:17.990671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:17.990671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:17.990671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:17.990671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:17.994090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:17.994090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:17.994090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:17.994090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:17.994090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:17.994090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:17.994090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:17.994090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:17.994090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:17.994090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:17.994090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:30:17.994090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:17.996821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 147/288] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 648 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:18.000826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb13c7-9db5-44bf-8555-d7b3db43ed77 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:18.025246 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f9836-d58f-4b99-9805-0703ca7d476a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:18.026379 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f9836-d58f-4b99-9805-0703ca7d476a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:18.026986 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f9836-d58f-4b99-9805-0703ca7d476a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:18.027257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f9836-d58f-4b99-9805-0703ca7d476a None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:18.041453 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f9836-d58f-4b99-9805-0703ca7d476a None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:18.044351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f9836-d58f-4b99-9805-0703ca7d476a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:18.056301 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 142/289] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:17 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 587 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:18.066353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f9836-d58f-4b99-9805-0703ca7d476a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:18.075664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:18.290773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb13c7-9db5-44bf-8555-d7b3db43ed77 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:18.432503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbac674-34e8-4739-ba2f-5ef2361ceed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:18.437527 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbac674-34e8-4739-ba2f-5ef2361ceed7 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:18.440042 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbac674-34e8-4739-ba2f-5ef2361ceed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:18.440042 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbac674-34e8-4739-ba2f-5ef2361ceed7 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:18.440418 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbac674-34e8-4739-ba2f-5ef2361ceed7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:18.444028 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbac674-34e8-4739-ba2f-5ef2361ceed7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:18.444028 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbac674-34e8-4739-ba2f-5ef2361ceed7 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:18.475669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f9836-d58f-4b99-9805-0703ca7d476a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:18.491819 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 148/290] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 495 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:18.498930 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbac674-34e8-4739-ba2f-5ef2361ceed7 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:18.652049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2039cbea-044e-4678-a83d-1b298e050e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:18.652049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2039cbea-044e-4678-a83d-1b298e050e3b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:18.652049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2039cbea-044e-4678-a83d-1b298e050e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:18.652049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2039cbea-044e-4678-a83d-1b298e050e3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:18.652941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2039cbea-044e-4678-a83d-1b298e050e3b None admin] RBAC: Adding query filter params (role.id=4eebd8098ab44bfc84cd3de29a679bdf, scope.project.id=b994c620a91d441aa453e5be1b958864, user.id=5b475a64338649da996a33f42448bb5b) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:18.653211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2039cbea-044e-4678-a83d-1b298e050e3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:18.656042 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2039cbea-044e-4678-a83d-1b298e050e3b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:18.664798 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:18.701590 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 143/291] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:18 2022] GET /identity/v3/role_assignments?user.id=5b475a64338649da996a33f42448bb5b&scope.project.id=b994c620a91d441aa453e5be1b958864&role.id=4eebd8098ab44bfc84cd3de29a679bdf => generated 608 bytes in 636 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:19.019545 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbac674-34e8-4739-ba2f-5ef2361ceed7 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:19.104715 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ac68a-f6c5-4996-bed9-66012475ca29 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:19.105502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ac68a-f6c5-4996-bed9-66012475ca29 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:19.105502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ac68a-f6c5-4996-bed9-66012475ca29 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:19.106643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 144/292] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:30:19.113711 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2498f1-7c24-4dfe-94f0-b84c06885c18 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:19.114090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2498f1-7c24-4dfe-94f0-b84c06885c18 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:19.114392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2498f1-7c24-4dfe-94f0-b84c06885c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:19.151318 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede18e2f-fa75-480e-b9ee-c11ec78d7f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:19.154250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede18e2f-fa75-480e-b9ee-c11ec78d7f62 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:19.155040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede18e2f-fa75-480e-b9ee-c11ec78d7f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:19.155887 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede18e2f-fa75-480e-b9ee-c11ec78d7f62 None admin] PATH_INFO: `/v3/users/de0075f3771b49cdb643ab2fbbc2dfd1` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:19.162954 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede18e2f-fa75-480e-b9ee-c11ec78d7f62 None admin] RBAC: Authorizing `identity:get_user(user_id=de0075f3771b49cdb643ab2fbbc2dfd1)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:19.166836 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede18e2f-fa75-480e-b9ee-c11ec78d7f62 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:19.170906 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 149/293] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:18 2022] GET /identity/v3/users/de0075f3771b49cdb643ab2fbbc2dfd1 => generated 297 bytes in 673 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:19.175326 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4f2498f1-7c24-4dfe-94f0-b84c06885c18 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:19.175326 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4f2498f1-7c24-4dfe-94f0-b84c06885c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:19.176478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede18e2f-fa75-480e-b9ee-c11ec78d7f62 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:19.326483 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede18e2f-fa75-480e-b9ee-c11ec78d7f62 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:19.344730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:19.411800 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebff84-fc2b-4414-8a3e-fa69fd979a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:19.416945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebff84-fc2b-4414-8a3e-fa69fd979a22 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:19.417772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebff84-fc2b-4414-8a3e-fa69fd979a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:19.418467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebff84-fc2b-4414-8a3e-fa69fd979a22 None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:19.421159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebff84-fc2b-4414-8a3e-fa69fd979a22 None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:19.424756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebff84-fc2b-4414-8a3e-fa69fd979a22 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:19.427239 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:19.429213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 150/294] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:19 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:19.435452 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebff84-fc2b-4414-8a3e-fa69fd979a22 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:19.482718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 145/295] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:30:19.588147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27ac7f-9941-45a8-be02-340ca0b0e52e None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:19.588147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27ac7f-9941-45a8-be02-340ca0b0e52e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:19.588147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27ac7f-9941-45a8-be02-340ca0b0e52e None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:19.590164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 146/296] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:19.596029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f067d89-cf65-4309-b5b7-9fd27fa7a61f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:19.596029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f067d89-cf65-4309-b5b7-9fd27fa7a61f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:19.596029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f067d89-cf65-4309-b5b7-9fd27fa7a61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:19.624034 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1f067d89-cf65-4309-b5b7-9fd27fa7a61f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:19.628039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1f067d89-cf65-4309-b5b7-9fd27fa7a61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:19.663913 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebff84-fc2b-4414-8a3e-fa69fd979a22 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:19.776117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec0fab-e4c8-4602-b7b7-b180e5ba3ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:19.776117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec0fab-e4c8-4602-b7b7-b180e5ba3ba9 None admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:19.776117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec0fab-e4c8-4602-b7b7-b180e5ba3ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:19.776117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec0fab-e4c8-4602-b7b7-b180e5ba3ba9 None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864/users/de0075f3771b49cdb643ab2fbbc2dfd1/roles/89f1b9db96764371860f845c00419e7a` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:19.784040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec0fab-e4c8-4602-b7b7-b180e5ba3ba9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b994c620a91d441aa453e5be1b958864, user_id=de0075f3771b49cdb643ab2fbbc2dfd1, role_id=89f1b9db96764371860f845c00419e7a)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:19.784040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec0fab-e4c8-4602-b7b7-b180e5ba3ba9 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:19.797634 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 147/297] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:19.826284 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 151/298] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:30:19 2022] PUT /identity/v3/projects/b994c620a91d441aa453e5be1b958864/users/de0075f3771b49cdb643ab2fbbc2dfd1/roles/89f1b9db96764371860f845c00419e7a => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:30:19.833332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f067d89-cf65-4309-b5b7-9fd27fa7a61f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:19.856627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fc4dd-0c3e-49c4-aefe-d42b3c364f5a None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:19.860042 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fc4dd-0c3e-49c4-aefe-d42b3c364f5a None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:19.860042 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fc4dd-0c3e-49c4-aefe-d42b3c364f5a None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:19.870735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 152/299] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:19 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:19.879377 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e2496-7408-43e6-87d1-105c4c1465d8 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:19.881113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e2496-7408-43e6-87d1-105c4c1465d8 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:19.881113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e2496-7408-43e6-87d1-105c4c1465d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:20.012277 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-e21e2496-7408-43e6-87d1-105c4c1465d8 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:20.015138 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e21e2496-7408-43e6-87d1-105c4c1465d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:20.071740 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f067d89-cf65-4309-b5b7-9fd27fa7a61f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:20.077872 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:20.107910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:20.145681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec7e64-5a2f-483f-85ab-822a30f09dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:20.146347 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec7e64-5a2f-483f-85ab-822a30f09dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:20.146638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec7e64-5a2f-483f-85ab-822a30f09dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:20.147235 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec7e64-5a2f-483f-85ab-822a30f09dc5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:20.165609 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec7e64-5a2f-483f-85ab-822a30f09dc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:20.172761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-20ec7e64-5a2f-483f-85ab-822a30f09dc5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:30:20.172761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:20.172761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:20.172761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:20.172761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:20.172761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:20.172761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:20.172761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:20.172761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:20.172761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:20.172761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:20.172761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:20.172761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:20.172761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:20.172761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:20.172761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:20.173693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:20.173693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:20.173693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:20.173693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:20.173693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:20.173693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:20.173693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:20.173693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:20.173693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:20.173693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:20.173693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:20.173693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:20.173693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:20.173693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:20.173693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:20.173693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:20.174481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:20.174481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:20.174481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:20.174481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:20.174481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:20.174481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:20.174481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:20.174481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:20.174481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:20.174481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:20.174481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:30:20.174481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:20.181616 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 148/300] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:20.183896 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec7e64-5a2f-483f-85ab-822a30f09dc5 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:20.187349 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 153/301] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:20.188943 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e2496-7408-43e6-87d1-105c4c1465d8 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:20.406066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec7e64-5a2f-483f-85ab-822a30f09dc5 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:20.502458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203d19e-dc6a-41b3-8a7d-05879b87fc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:20.503115 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203d19e-dc6a-41b3-8a7d-05879b87fc14 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:20.503386 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203d19e-dc6a-41b3-8a7d-05879b87fc14 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:20.503651 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203d19e-dc6a-41b3-8a7d-05879b87fc14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:20.517828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203d19e-dc6a-41b3-8a7d-05879b87fc14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:20.525654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-c203d19e-dc6a-41b3-8a7d-05879b87fc14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:20.525654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:20.525654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:20.525654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:20.525654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:20.525654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:20.525654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:20.525654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:20.525654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:20.525654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:20.525654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:20.525654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:20.525654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:20.525654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:20.525654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:20.525654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:20.526750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:20.526750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:20.526750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:20.526750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:20.526750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:20.526750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:20.526750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:20.526750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:20.526750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:20.526750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:20.526750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:20.526750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:20.526750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:20.526750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:20.526750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:20.526750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:20.528854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:20.528854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:20.528854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:20.528854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:20.528854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:20.528854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:20.528854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:20.528854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:20.528854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:20.528854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:20.528854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:20.528854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:20.528854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 149/302] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:20.534371 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203d19e-dc6a-41b3-8a7d-05879b87fc14 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:20.578007 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e2496-7408-43e6-87d1-105c4c1465d8 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:20.599174 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:20.637036 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb228513-4ef7-4281-abc5-cb6c862a8d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:20.638461 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb228513-4ef7-4281-abc5-cb6c862a8d9b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:20.639344 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb228513-4ef7-4281-abc5-cb6c862a8d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:20.639952 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb228513-4ef7-4281-abc5-cb6c862a8d9b None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:20.641023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb228513-4ef7-4281-abc5-cb6c862a8d9b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:20.641602 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb228513-4ef7-4281-abc5-cb6c862a8d9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:20.644011 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb228513-4ef7-4281-abc5-cb6c862a8d9b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:20.655851 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 154/303] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 468 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:20.657212 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb228513-4ef7-4281-abc5-cb6c862a8d9b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:20.667696 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203d19e-dc6a-41b3-8a7d-05879b87fc14 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:20.679674 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:20.715633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df20a9-4cb7-4bc4-a9e8-721244923955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:20.716880 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df20a9-4cb7-4bc4-a9e8-721244923955 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:20.717279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df20a9-4cb7-4bc4-a9e8-721244923955 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:20.717669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df20a9-4cb7-4bc4-a9e8-721244923955 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:20.719255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df20a9-4cb7-4bc4-a9e8-721244923955 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:20.721591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df20a9-4cb7-4bc4-a9e8-721244923955 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:20.728193 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 150/304] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:20.728193 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df20a9-4cb7-4bc4-a9e8-721244923955 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:20.787727 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb228513-4ef7-4281-abc5-cb6c862a8d9b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:20.840878 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a98f39-853b-4706-bdaa-ae24d3a7f917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:20.841667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a98f39-853b-4706-bdaa-ae24d3a7f917 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:20.842061 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a98f39-853b-4706-bdaa-ae24d3a7f917 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:20.842451 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a98f39-853b-4706-bdaa-ae24d3a7f917 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:20.843257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a98f39-853b-4706-bdaa-ae24d3a7f917 None admin] RBAC: Adding query filter params (name=member) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:20.843954 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a98f39-853b-4706-bdaa-ae24d3a7f917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:20.845681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a98f39-853b-4706-bdaa-ae24d3a7f917 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:20.864591 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 155/305] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:20.870444 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a98f39-853b-4706-bdaa-ae24d3a7f917 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:20.887442 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df20a9-4cb7-4bc4-a9e8-721244923955 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:20.919267 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:20.948503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b17c6-081d-4c79-bc91-a7a1d81799d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:20.949472 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b17c6-081d-4c79-bc91-a7a1d81799d2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:20.949751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b17c6-081d-4c79-bc91-a7a1d81799d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:20.950183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b17c6-081d-4c79-bc91-a7a1d81799d2 None admin] PATH_INFO: `/v3/users/6016bf7c58b045ddb36aa0efd2ef29fd` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:20.969140 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b17c6-081d-4c79-bc91-a7a1d81799d2 None admin] RBAC: Authorizing `identity:get_user(user_id=6016bf7c58b045ddb36aa0efd2ef29fd)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:20.970827 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b17c6-081d-4c79-bc91-a7a1d81799d2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:20.976592 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 151/306] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:20 2022] GET /identity/v3/users/6016bf7c58b045ddb36aa0efd2ef29fd => generated 293 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:20.976592 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b17c6-081d-4c79-bc91-a7a1d81799d2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:20.983000 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:21.010028 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a98f39-853b-4706-bdaa-ae24d3a7f917 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:21.069567 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f412247-496f-4674-89c0-0ffde87f1301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:21.070971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f412247-496f-4674-89c0-0ffde87f1301 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:21.071654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f412247-496f-4674-89c0-0ffde87f1301 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:21.072346 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f412247-496f-4674-89c0-0ffde87f1301 None admin] PATH_INFO: `/v3/groups` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:21.073731 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f412247-496f-4674-89c0-0ffde87f1301 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:21.075374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f412247-496f-4674-89c0-0ffde87f1301 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:21.101543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 156/307] 10.222.0.25 () {60 vars in 1134 bytes} [Tue Aug 30 14:30:20 2022] POST /identity/v3/groups => generated 229 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:21.102623 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f412247-496f-4674-89c0-0ffde87f1301 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:21.143714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b17c6-081d-4c79-bc91-a7a1d81799d2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:21.232266 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fea14d-62cc-4d74-9ae0-65a0763366ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:21.233746 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fea14d-62cc-4d74-9ae0-65a0763366ed None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:21.235071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fea14d-62cc-4d74-9ae0-65a0763366ed None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:21.236064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fea14d-62cc-4d74-9ae0-65a0763366ed None admin] PATH_INFO: `/v3/users/cda0c9596f0347198afa747d65b1c505` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:21.239249 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fea14d-62cc-4d74-9ae0-65a0763366ed None admin] RBAC: Authorizing `identity:get_user(user_id=cda0c9596f0347198afa747d65b1c505)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:21.242822 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fea14d-62cc-4d74-9ae0-65a0763366ed None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:21.247404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 152/308] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:20 2022] GET /identity/v3/users/cda0c9596f0347198afa747d65b1c505 => generated 293 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:21.259097 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fea14d-62cc-4d74-9ae0-65a0763366ed None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:21.311893 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f412247-496f-4674-89c0-0ffde87f1301 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:21.334681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:21.373369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f224d-911c-4b16-a484-3110609c6387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:21.373369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f224d-911c-4b16-a484-3110609c6387 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:21.373799 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f224d-911c-4b16-a484-3110609c6387 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:21.373926 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f224d-911c-4b16-a484-3110609c6387 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:21.375416 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f224d-911c-4b16-a484-3110609c6387 None admin] RBAC: Adding query filter params (role.id=89f1b9db96764371860f845c00419e7a, user.id=6016bf7c58b045ddb36aa0efd2ef29fd, scope.system=all) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:21.375941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f224d-911c-4b16-a484-3110609c6387 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:21.378666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f224d-911c-4b16-a484-3110609c6387 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:21.386516 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 157/309] 10.222.0.25 () {56 vars in 1308 bytes} [Tue Aug 30 14:30:21 2022] GET /identity/v3/role_assignments?user.id=6016bf7c58b045ddb36aa0efd2ef29fd&scope.system=all&role.id=89f1b9db96764371860f845c00419e7a => generated 226 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:21.416879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fea14d-62cc-4d74-9ae0-65a0763366ed None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:21.457301 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaef43-1f50-49aa-9836-b36a93b0521b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:21.458294 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaef43-1f50-49aa-9836-b36a93b0521b None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:21.459100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaef43-1f50-49aa-9836-b36a93b0521b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:21.459526 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaef43-1f50-49aa-9836-b36a93b0521b None admin] PATH_INFO: `/v3/system/users/cda0c9596f0347198afa747d65b1c505/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:21.461253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beaef43-1f50-49aa-9836-b36a93b0521b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cda0c9596f0347198afa747d65b1c505, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:21.462801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beaef43-1f50-49aa-9836-b36a93b0521b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:21.474473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 153/310] 10.222.0.25 () {58 vars in 1334 bytes} [Tue Aug 30 14:30:21 2022] PUT /identity/v3/system/users/cda0c9596f0347198afa747d65b1c505/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:30:22.197672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b9ba8-fc35-44a9-b523-badb655ca366 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:22.198626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b9ba8-fc35-44a9-b523-badb655ca366 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:22.198626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b9ba8-fc35-44a9-b523-badb655ca366 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:22.200949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 158/311] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:22.210405 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35213173-63c0-4c53-b239-8e220286ed95 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:22.211889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35213173-63c0-4c53-b239-8e220286ed95 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:22.212708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35213173-63c0-4c53-b239-8e220286ed95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:22.244125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-35213173-63c0-4c53-b239-8e220286ed95 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:22.244125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-35213173-63c0-4c53-b239-8e220286ed95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:22.319228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 154/312] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:30:22.596989 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b9ba8-fc35-44a9-b523-badb655ca366 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:22.600964 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:22.782984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b9ba8-fc35-44a9-b523-badb655ca366 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:22.822522 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5324b-ccc6-40b8-8146-9d61e8e6ac89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:22.823251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5324b-ccc6-40b8-8146-9d61e8e6ac89 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:22.823528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5324b-ccc6-40b8-8146-9d61e8e6ac89 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:22.824037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5324b-ccc6-40b8-8146-9d61e8e6ac89 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:22.837078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5324b-ccc6-40b8-8146-9d61e8e6ac89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:22.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-92a5324b-ccc6-40b8-8146-9d61e8e6ac89 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:30:22.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:22.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:22.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:22.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:22.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:22.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:22.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:22.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:22.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:22.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:22.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:22.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:22.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:22.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:22.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:22.843286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:22.843286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:22.843286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:22.843286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:22.843286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:22.843286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:22.843286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:22.843286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:22.843286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:22.843286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:22.843286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:22.843286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:22.843286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:22.843286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:22.843286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:22.843286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:22.844467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:22.844467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:22.844467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:22.844467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:22.844467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:22.844467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:22.844467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:22.844467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:22.844467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:22.844467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:22.844467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:30:22.844467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:22.844467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 159/313] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:22.853989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35213173-63c0-4c53-b239-8e220286ed95 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:22.908764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:22.979383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35213173-63c0-4c53-b239-8e220286ed95 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:23.017478 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc181c-163f-4523-83e1-239fec28b752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:23.018503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc181c-163f-4523-83e1-239fec28b752 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:23.018983 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc181c-163f-4523-83e1-239fec28b752 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:23.019459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc181c-163f-4523-83e1-239fec28b752 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:23.020370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc181c-163f-4523-83e1-239fec28b752 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:23.020889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc181c-163f-4523-83e1-239fec28b752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:23.022531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc181c-163f-4523-83e1-239fec28b752 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:23.033392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 155/314] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:23.038915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5324b-ccc6-40b8-8146-9d61e8e6ac89 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:23.163330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:23.211674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887719a-4bc2-4bf9-b23a-69caf7d98c4c None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:23.212250 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887719a-4bc2-4bf9-b23a-69caf7d98c4c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:23.212250 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887719a-4bc2-4bf9-b23a-69caf7d98c4c None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:23.215110 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5324b-ccc6-40b8-8146-9d61e8e6ac89 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:23.218142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 156/315] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:30:23.219902 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93ff27-eecb-421f-8f1d-adc97a2f97eb None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:23.220470 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93ff27-eecb-421f-8f1d-adc97a2f97eb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:23.220925 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93ff27-eecb-421f-8f1d-adc97a2f97eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:23.240966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-cb93ff27-eecb-421f-8f1d-adc97a2f97eb None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:23.242856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cb93ff27-eecb-421f-8f1d-adc97a2f97eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:23.255171 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b5c26-6f8f-4ea9-8e5c-71f6334f1a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:23.255816 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b5c26-6f8f-4ea9-8e5c-71f6334f1a94 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:23.256122 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b5c26-6f8f-4ea9-8e5c-71f6334f1a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:23.256393 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b5c26-6f8f-4ea9-8e5c-71f6334f1a94 None admin] PATH_INFO: `/v3/users/de0075f3771b49cdb643ab2fbbc2dfd1` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:23.257605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b5c26-6f8f-4ea9-8e5c-71f6334f1a94 None admin] RBAC: Authorizing `identity:get_user(user_id=de0075f3771b49cdb643ab2fbbc2dfd1)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:23.259181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b5c26-6f8f-4ea9-8e5c-71f6334f1a94 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:23.261137 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 160/316] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:23 2022] GET /identity/v3/users/de0075f3771b49cdb643ab2fbbc2dfd1 => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:23.266904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b5c26-6f8f-4ea9-8e5c-71f6334f1a94 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:23.346957 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 157/317] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:23.436346 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b5c26-6f8f-4ea9-8e5c-71f6334f1a94 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:23.477706 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:23.483679 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc562dc3-a891-40fd-87ad-73ac7d905577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:23.484672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc562dc3-a891-40fd-87ad-73ac7d905577 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:23.485173 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc562dc3-a891-40fd-87ad-73ac7d905577 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:23.485456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc562dc3-a891-40fd-87ad-73ac7d905577 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:23.562242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc562dc3-a891-40fd-87ad-73ac7d905577 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:23.564105 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc562dc3-a891-40fd-87ad-73ac7d905577 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:23.655512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93ff27-eecb-421f-8f1d-adc97a2f97eb None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:23.676802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 161/318] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:23.685679 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc562dc3-a891-40fd-87ad-73ac7d905577 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:23.700176 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:23.736973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:23.799315 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93ff27-eecb-421f-8f1d-adc97a2f97eb None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:23.840552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f83bea-f2f3-4a19-9419-d533b974766f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:23.841503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f83bea-f2f3-4a19-9419-d533b974766f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:23.841803 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f83bea-f2f3-4a19-9419-d533b974766f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:23.842083 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f83bea-f2f3-4a19-9419-d533b974766f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:23.843194 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f83bea-f2f3-4a19-9419-d533b974766f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:23.844878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f83bea-f2f3-4a19-9419-d533b974766f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:23.846669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 158/319] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:23.847566 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7997a57-e757-4538-a1c8-6c125465a68e None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:23.847834 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7997a57-e757-4538-a1c8-6c125465a68e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:23.848150 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7997a57-e757-4538-a1c8-6c125465a68e None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:23.849570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 159/320] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:23.851807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc562dc3-a891-40fd-87ad-73ac7d905577 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:23.854384 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7997a57-e757-4538-a1c8-6c125465a68e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:23.893976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f5777-c61e-4298-98ed-be8ae9b6a1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:23.894626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f5777-c61e-4298-98ed-be8ae9b6a1ca None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:23.894904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f5777-c61e-4298-98ed-be8ae9b6a1ca None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:23.895173 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f5777-c61e-4298-98ed-be8ae9b6a1ca None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:23.896788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f5777-c61e-4298-98ed-be8ae9b6a1ca None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:23.898287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f5777-c61e-4298-98ed-be8ae9b6a1ca None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:23.900395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 162/321] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:23 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:23.901615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed9db3-f3e6-49d5-a3dc-e0fb0c2571fc None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:23.901888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed9db3-f3e6-49d5-a3dc-e0fb0c2571fc None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:23.902156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed9db3-f3e6-49d5-a3dc-e0fb0c2571fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:23.920984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-95ed9db3-f3e6-49d5-a3dc-e0fb0c2571fc None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:23.922370 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-95ed9db3-f3e6-49d5-a3dc-e0fb0c2571fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:23.996676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:23.998482 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 163/322] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:23.999908 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bfe92-387e-4087-aa00-7a46a34746c0 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:24.000238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bfe92-387e-4087-aa00-7a46a34746c0 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:24.000504 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bfe92-387e-4087-aa00-7a46a34746c0 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:24.001913 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 164/323] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:24.002539 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0bfe92-387e-4087-aa00-7a46a34746c0 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:24.018384 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7997a57-e757-4538-a1c8-6c125465a68e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:24.051179 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:24.064657 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d073320-33a9-4934-9525-79b61e99cd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:24.065612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d073320-33a9-4934-9525-79b61e99cd12 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:24.066045 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d073320-33a9-4934-9525-79b61e99cd12 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:24.066469 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d073320-33a9-4934-9525-79b61e99cd12 None admin] PATH_INFO: `/v3/groups` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:24.067508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d073320-33a9-4934-9525-79b61e99cd12 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:24.069787 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d073320-33a9-4934-9525-79b61e99cd12 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:24.096058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 160/324] 10.222.0.25 () {60 vars in 1134 bytes} [Tue Aug 30 14:30:23 2022] POST /identity/v3/groups => generated 226 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:24.096058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e24311-ce7a-4fd1-a0dd-68b5bd4cd051 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:24.096058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e24311-ce7a-4fd1-a0dd-68b5bd4cd051 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:24.096058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e24311-ce7a-4fd1-a0dd-68b5bd4cd051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:24.136035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d1e24311-ce7a-4fd1-a0dd-68b5bd4cd051 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:24.140035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d1e24311-ce7a-4fd1-a0dd-68b5bd4cd051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:24.156523 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0bfe92-387e-4087-aa00-7a46a34746c0 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:24.216891 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7afd82-0310-430f-8616-74cbf948ee58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:24.217975 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7afd82-0310-430f-8616-74cbf948ee58 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:24.218539 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7afd82-0310-430f-8616-74cbf948ee58 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:24.219224 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7afd82-0310-430f-8616-74cbf948ee58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:24.220854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7afd82-0310-430f-8616-74cbf948ee58 None admin] RBAC: Adding query filter params (role.id=89f1b9db96764371860f845c00419e7a, scope.project.id=b994c620a91d441aa453e5be1b958864, user.id=de0075f3771b49cdb643ab2fbbc2dfd1) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:24.225706 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7afd82-0310-430f-8616-74cbf948ee58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:24.227469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7afd82-0310-430f-8616-74cbf948ee58 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:24.240727 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 165/325] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:24 2022] GET /identity/v3/role_assignments?user.id=de0075f3771b49cdb643ab2fbbc2dfd1&scope.project.id=b994c620a91d441aa453e5be1b958864&role.id=89f1b9db96764371860f845c00419e7a => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:24.260752 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 161/326] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:24.283171 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7afd82-0310-430f-8616-74cbf948ee58 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:24.424753 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:24.434504 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7afd82-0310-430f-8616-74cbf948ee58 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:24.481489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf2caa-3eef-4874-98f8-ac38a030ba3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:24.482531 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf2caa-3eef-4874-98f8-ac38a030ba3d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:24.483008 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf2caa-3eef-4874-98f8-ac38a030ba3d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:24.483463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf2caa-3eef-4874-98f8-ac38a030ba3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:24.496633 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf2caa-3eef-4874-98f8-ac38a030ba3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:24.502335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-bbbf2caa-3eef-4874-98f8-ac38a030ba3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:24.502335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:24.502335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:24.502335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:24.502335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:24.502335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:24.502335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:24.502335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:24.502335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:24.502335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:24.502335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:24.502335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:24.502335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:24.502335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:24.502335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:24.502335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:24.503278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:24.503278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:24.503278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:24.503278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:24.503278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:24.503278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:24.503278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:24.503278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:24.503278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:24.503278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:24.503278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:24.503278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:24.503278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:24.503278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:24.503278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:24.503278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:24.505627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:24.505627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:24.505627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:24.505627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:24.505627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:24.505627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:24.505627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:24.505627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:24.505627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:24.505627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:24.505627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:24.505627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:24.505627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 166/327] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:24.509710 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e24311-ce7a-4fd1-a0dd-68b5bd4cd051 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:24.536174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf2caa-3eef-4874-98f8-ac38a030ba3d None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:24.648034 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:24.671044 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e24311-ce7a-4fd1-a0dd-68b5bd4cd051 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:24.700839 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf2caa-3eef-4874-98f8-ac38a030ba3d None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:24.719894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcdf69-ff0d-4e0b-ad14-717fe8f032f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:24.720854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcdf69-ff0d-4e0b-ad14-717fe8f032f9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:24.721275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcdf69-ff0d-4e0b-ad14-717fe8f032f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:24.721677 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcdf69-ff0d-4e0b-ad14-717fe8f032f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:24.722496 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebcdf69-ff0d-4e0b-ad14-717fe8f032f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:24.722948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebcdf69-ff0d-4e0b-ad14-717fe8f032f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:24.725331 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebcdf69-ff0d-4e0b-ad14-717fe8f032f9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:24.735766 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 162/328] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:24.741138 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcdf69-ff0d-4e0b-ad14-717fe8f032f9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:24.759500 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34146e13-43c8-4088-b75c-e6e7bd616494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:24.760248 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34146e13-43c8-4088-b75c-e6e7bd616494 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:24.760525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34146e13-43c8-4088-b75c-e6e7bd616494 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:24.760795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34146e13-43c8-4088-b75c-e6e7bd616494 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:24.796929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34146e13-43c8-4088-b75c-e6e7bd616494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:24.802398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-34146e13-43c8-4088-b75c-e6e7bd616494 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:30:24.802398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:24.802398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:24.802398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:24.802398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:24.802398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:24.802398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:24.802398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:24.802398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:24.802398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:24.802398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:24.802398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:24.802398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:24.802398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:24.802398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:24.802398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:24.803357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:24.803357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:24.803357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:24.803357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:24.803357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:24.803357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:24.803357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:24.803357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:24.803357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:24.803357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:24.803357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:24.803357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:24.803357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:24.803357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:24.803357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:24.803357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:24.804242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:24.804242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:24.804242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:24.804242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:24.804242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:24.804242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:24.804242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:24.804242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:24.804242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:24.804242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:24.804242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:30:24.804242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:24.804242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 167/329] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:24.811516 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34146e13-43c8-4088-b75c-e6e7bd616494 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:24.847983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:24.908427 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcdf69-ff0d-4e0b-ad14-717fe8f032f9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:24.962378 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad435d89-b02a-481b-b8d4-46cf33ebcec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:24.963070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad435d89-b02a-481b-b8d4-46cf33ebcec6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:24.963346 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad435d89-b02a-481b-b8d4-46cf33ebcec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:24.963614 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad435d89-b02a-481b-b8d4-46cf33ebcec6 None admin] PATH_INFO: `/v3/users/cda0c9596f0347198afa747d65b1c505` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:24.964829 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad435d89-b02a-481b-b8d4-46cf33ebcec6 None admin] RBAC: Authorizing `identity:get_user(user_id=cda0c9596f0347198afa747d65b1c505)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:24.966299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad435d89-b02a-481b-b8d4-46cf33ebcec6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:24.968306 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 163/330] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:24 2022] GET /identity/v3/users/cda0c9596f0347198afa747d65b1c505 => generated 293 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:24.974478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34146e13-43c8-4088-b75c-e6e7bd616494 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:24.980141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad435d89-b02a-481b-b8d4-46cf33ebcec6 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:25.023635 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:25.029887 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b4807-3b64-4ec3-9b2c-e68b85439794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:25.030599 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b4807-3b64-4ec3-9b2c-e68b85439794 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:25.031043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b4807-3b64-4ec3-9b2c-e68b85439794 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:25.031566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b4807-3b64-4ec3-9b2c-e68b85439794 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:25.033575 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b4807-3b64-4ec3-9b2c-e68b85439794 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:25.034046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b4807-3b64-4ec3-9b2c-e68b85439794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:25.035980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b4807-3b64-4ec3-9b2c-e68b85439794 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:25.049547 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 168/331] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:25.055749 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b4807-3b64-4ec3-9b2c-e68b85439794 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:25.241578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad435d89-b02a-481b-b8d4-46cf33ebcec6 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:25.253113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:25.286062 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b4807-3b64-4ec3-9b2c-e68b85439794 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:25.332149 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29703b0-e352-4315-b25d-21342b813c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:25.333093 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29703b0-e352-4315-b25d-21342b813c9a None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:25.334463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29703b0-e352-4315-b25d-21342b813c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:25.335087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29703b0-e352-4315-b25d-21342b813c9a None admin] PATH_INFO: `/v3/users/6016bf7c58b045ddb36aa0efd2ef29fd` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:25.337255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29703b0-e352-4315-b25d-21342b813c9a None admin] RBAC: Authorizing `identity:get_user(user_id=6016bf7c58b045ddb36aa0efd2ef29fd)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:25.342083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29703b0-e352-4315-b25d-21342b813c9a None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:25.350644 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 169/332] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:25 2022] GET /identity/v3/users/6016bf7c58b045ddb36aa0efd2ef29fd => generated 293 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:25.355883 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29703b0-e352-4315-b25d-21342b813c9a None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:25.362187 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f7c6b-7b53-405b-88e5-cf07ebe44a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:25.366733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f7c6b-7b53-405b-88e5-cf07ebe44a10 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:25.367264 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f7c6b-7b53-405b-88e5-cf07ebe44a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:25.367887 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f7c6b-7b53-405b-88e5-cf07ebe44a10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:25.369554 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f7c6b-7b53-405b-88e5-cf07ebe44a10 None admin] RBAC: Adding query filter params (role.id=4eebd8098ab44bfc84cd3de29a679bdf, user.id=cda0c9596f0347198afa747d65b1c505, scope.system=all) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:25.371320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f7c6b-7b53-405b-88e5-cf07ebe44a10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:25.376795 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f7c6b-7b53-405b-88e5-cf07ebe44a10 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:25.414408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 164/333] 10.222.0.25 () {56 vars in 1308 bytes} [Tue Aug 30 14:30:24 2022] GET /identity/v3/role_assignments?user.id=cda0c9596f0347198afa747d65b1c505&scope.system=all&role.id=4eebd8098ab44bfc84cd3de29a679bdf => generated 510 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:25.514725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29703b0-e352-4315-b25d-21342b813c9a None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:25.560696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5153dd-cf21-45c9-a26b-f0e09b83591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:25.561407 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5153dd-cf21-45c9-a26b-f0e09b83591c None admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:25.561687 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5153dd-cf21-45c9-a26b-f0e09b83591c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:25.561965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5153dd-cf21-45c9-a26b-f0e09b83591c None admin] PATH_INFO: `/v3/system/users/6016bf7c58b045ddb36aa0efd2ef29fd/roles/89f1b9db96764371860f845c00419e7a` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:25.563543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5153dd-cf21-45c9-a26b-f0e09b83591c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6016bf7c58b045ddb36aa0efd2ef29fd, role_id=89f1b9db96764371860f845c00419e7a)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:25.568029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5153dd-cf21-45c9-a26b-f0e09b83591c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:25.582539 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 170/334] 10.222.0.25 () {58 vars in 1334 bytes} [Tue Aug 30 14:30:25 2022] PUT /identity/v3/system/users/6016bf7c58b045ddb36aa0efd2ef29fd/roles/89f1b9db96764371860f845c00419e7a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:30:27.193993 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6451ac-9faf-4c75-974f-59b713e0ddab None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:27.194571 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6451ac-9faf-4c75-974f-59b713e0ddab None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:27.194571 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6451ac-9faf-4c75-974f-59b713e0ddab None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:27.196372 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 165/335] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:27.202522 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2338cc1-0998-4c62-a063-1cc2e56a219b None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:27.203548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2338cc1-0998-4c62-a063-1cc2e56a219b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:27.204065 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2338cc1-0998-4c62-a063-1cc2e56a219b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:27.231116 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-b2338cc1-0998-4c62-a063-1cc2e56a219b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:27.232759 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b2338cc1-0998-4c62-a063-1cc2e56a219b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:27.237002 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:27.317411 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360df82-cfa2-4933-bfc2-a343c894bc33 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:27.317717 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360df82-cfa2-4933-bfc2-a343c894bc33 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:27.318135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360df82-cfa2-4933-bfc2-a343c894bc33 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:27.319827 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 166/336] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:27.332230 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc4273-9faf-46b4-8824-6060d0311884 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:27.332570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc4273-9faf-46b4-8824-6060d0311884 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:27.332788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc4273-9faf-46b4-8824-6060d0311884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:27.361115 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 171/337] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:27.381307 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-13dc4273-9faf-46b4-8824-6060d0311884 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:27.382555 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-13dc4273-9faf-46b4-8824-6060d0311884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:27.494784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 167/338] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:27.824292 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2902b1-29dd-493c-83c2-5a5ad293a4fc None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:27.824490 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2902b1-29dd-493c-83c2-5a5ad293a4fc None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:27.824751 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2902b1-29dd-493c-83c2-5a5ad293a4fc None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:27.826352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 172/339] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:27.832724 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1cc3c-ae2f-4862-962a-462861e22595 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:27.833745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1cc3c-ae2f-4862-962a-462861e22595 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:27.834210 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1cc3c-ae2f-4862-962a-462861e22595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:27.848582 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:27.868469 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1cb1cc3c-ae2f-4862-962a-462861e22595 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:27.870036 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1cb1cc3c-ae2f-4862-962a-462861e22595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:27.894812 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2902b1-29dd-493c-83c2-5a5ad293a4fc None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:28.010731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 168/340] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:28.099386 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1cc3c-ae2f-4862-962a-462861e22595 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:28.151213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2902b1-29dd-493c-83c2-5a5ad293a4fc None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:28.236229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe99c3-a43b-4980-b6a6-a040544bdd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:28.236894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe99c3-a43b-4980-b6a6-a040544bdd4d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:28.237981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe99c3-a43b-4980-b6a6-a040544bdd4d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:28.237981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe99c3-a43b-4980-b6a6-a040544bdd4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:28.257714 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:28.276793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe99c3-a43b-4980-b6a6-a040544bdd4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:28.282268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-8abe99c3-a43b-4980-b6a6-a040544bdd4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:28.282268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:28.282268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:28.282268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:28.282268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:28.282268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:28.282268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:28.282268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:28.282268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:28.282268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:28.282268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:28.282268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:28.282268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:28.282268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:28.282268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:28.282268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:28.283214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:28.283214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:28.283214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:28.283214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:28.283214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:28.283214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:28.283214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:28.283214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:28.283214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:28.283214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:28.283214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:28.283214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:28.283214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:28.283214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:28.283214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:28.283214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:28.284480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:28.284480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:28.284480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:28.284480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:28.284480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:28.284480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:28.284480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:28.284480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:28.284480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:28.284480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:28.284480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:28.284480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:28.284480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 173/341] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:28.293085 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe99c3-a43b-4980-b6a6-a040544bdd4d None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:28.295925 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1cc3c-ae2f-4862-962a-462861e22595 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:28.351551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:28.365100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be22f95-3df4-4750-80cc-56e169aaacd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:28.365764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be22f95-3df4-4750-80cc-56e169aaacd3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:28.366044 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be22f95-3df4-4750-80cc-56e169aaacd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:28.366318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be22f95-3df4-4750-80cc-56e169aaacd3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:28.386831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be22f95-3df4-4750-80cc-56e169aaacd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:28.392460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-8be22f95-3df4-4750-80cc-56e169aaacd3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:28.392460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:28.392460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:28.392460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:28.392460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:28.392460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:28.392460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:28.392460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:28.392460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:28.392460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:28.392460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:28.392460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:28.392460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:28.392460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:28.392460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:28.392460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:28.393416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:28.393416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:28.393416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:28.393416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:28.393416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:28.393416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:28.393416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:28.393416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:28.393416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:28.393416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:28.393416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:28.393416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:28.393416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:28.393416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:28.393416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:28.393416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:28.394211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:28.394211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:28.394211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:28.394211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:28.394211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:28.394211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:28.394211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:28.394211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:28.394211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:28.394211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:28.394211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:28.394211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:28.394211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 169/342] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:28.401763 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be22f95-3df4-4750-80cc-56e169aaacd3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:28.604541 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be22f95-3df4-4750-80cc-56e169aaacd3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:28.612067 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe99c3-a43b-4980-b6a6-a040544bdd4d None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:28.674037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7205f5-7a49-4bd0-a0b2-00a20d6f1caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:28.674275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7205f5-7a49-4bd0-a0b2-00a20d6f1caa None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:28.680027 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7205f5-7a49-4bd0-a0b2-00a20d6f1caa None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:28.680027 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7205f5-7a49-4bd0-a0b2-00a20d6f1caa None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:28.680027 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7205f5-7a49-4bd0-a0b2-00a20d6f1caa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:28.680379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab0153-8e0f-4d87-8353-c63c0f9adfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:28.680481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7205f5-7a49-4bd0-a0b2-00a20d6f1caa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:28.680563 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab0153-8e0f-4d87-8353-c63c0f9adfca None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:28.680563 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab0153-8e0f-4d87-8353-c63c0f9adfca None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:28.680563 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab0153-8e0f-4d87-8353-c63c0f9adfca None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:28.680782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7205f5-7a49-4bd0-a0b2-00a20d6f1caa None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:28.680859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab0153-8e0f-4d87-8353-c63c0f9adfca None admin] RBAC: Adding query filter params (name=member) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:28.680859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab0153-8e0f-4d87-8353-c63c0f9adfca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:28.680859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab0153-8e0f-4d87-8353-c63c0f9adfca None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:28.692288 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 174/343] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:28.694099 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab0153-8e0f-4d87-8353-c63c0f9adfca None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:28.694311 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 170/344] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:28.695843 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c6eaf-4e8c-4750-a7e6-b856801aeedc None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:28.696367 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c6eaf-4e8c-4750-a7e6-b856801aeedc None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:28.696783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c6eaf-4e8c-4750-a7e6-b856801aeedc None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:28.700227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 171/345] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:28.701380 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c6eaf-4e8c-4750-a7e6-b856801aeedc None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:28.797390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:28.812740 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:28.883779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab0153-8e0f-4d87-8353-c63c0f9adfca None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:28.884497 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c6eaf-4e8c-4750-a7e6-b856801aeedc None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:28.939812 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd0ec7-25ae-4011-8039-13e347b77017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:28.940473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd0ec7-25ae-4011-8039-13e347b77017 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:28.940747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd0ec7-25ae-4011-8039-13e347b77017 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:28.941012 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd0ec7-25ae-4011-8039-13e347b77017 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:28.959781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b5ad4-f5c8-4e1e-ae49-341d0305ba10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:28.960160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b5ad4-f5c8-4e1e-ae49-341d0305ba10 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:28.960376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b5ad4-f5c8-4e1e-ae49-341d0305ba10 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:28.961099 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b5ad4-f5c8-4e1e-ae49-341d0305ba10 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:28.982150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b5ad4-f5c8-4e1e-ae49-341d0305ba10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:28.992829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-439b5ad4-f5c8-4e1e-ae49-341d0305ba10 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:28.992829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:28.992829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:28.992829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:28.992829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:28.992829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:28.992829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:28.992829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:28.992829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:28.992829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:28.992829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:28.992829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:28.992829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:28.992829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:28.992829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:28.992829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:28.993924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:28.993924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:28.993924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:28.993924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:28.993924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:28.993924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:28.993924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:28.993924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:28.993924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:28.993924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:28.993924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:28.993924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:28.993924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:28.993924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:28.993924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:28.993924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:28.994795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:28.994795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:28.994795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:28.994795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:28.994795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:28.994795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:28.994795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:28.994795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:28.994795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:28.994795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:28.994795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:28.994795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:28.995427 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 175/346] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:29.001150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b5ad4-f5c8-4e1e-ae49-341d0305ba10 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:29.054552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd0ec7-25ae-4011-8039-13e347b77017 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:29.056388 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd0ec7-25ae-4011-8039-13e347b77017 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:29.168360 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:29.181693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b5ad4-f5c8-4e1e-ae49-341d0305ba10 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:29.226923 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:29.254027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800c297-aead-4070-af77-5db46ece4f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:29.254799 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800c297-aead-4070-af77-5db46ece4f6e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:29.254960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800c297-aead-4070-af77-5db46ece4f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:29.255236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800c297-aead-4070-af77-5db46ece4f6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:29.274037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 172/347] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:29.276714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b891b0-a7dd-4d50-a0bd-d1773f35d769 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:29.277191 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b891b0-a7dd-4d50-a0bd-d1773f35d769 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:29.277660 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b891b0-a7dd-4d50-a0bd-d1773f35d769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:29.305564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d2b891b0-a7dd-4d50-a0bd-d1773f35d769 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:29.310220 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d2b891b0-a7dd-4d50-a0bd-d1773f35d769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:29.368415 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5800c297-aead-4070-af77-5db46ece4f6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:29.371348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5800c297-aead-4070-af77-5db46ece4f6e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:29.424852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 173/348] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:29.424852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2612ec-302e-434f-959f-fbd1c06a6196 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:29.424852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2612ec-302e-434f-959f-fbd1c06a6196 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:29.424852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2612ec-302e-434f-959f-fbd1c06a6196 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:29.426925 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 174/349] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:29.431419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2612ec-302e-434f-959f-fbd1c06a6196 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:29.570074 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 176/350] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:29.571781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70221d38-b44c-43fc-a42d-7095daac5721 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:29.572378 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70221d38-b44c-43fc-a42d-7095daac5721 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:29.572854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70221d38-b44c-43fc-a42d-7095daac5721 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:29.572959 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:29.574836 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 177/351] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:29.575875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70221d38-b44c-43fc-a42d-7095daac5721 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:29.612888 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2612ec-302e-434f-959f-fbd1c06a6196 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:29.618250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:29.690755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f151a-956c-4dbc-a265-412d30de204a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:29.691427 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f151a-956c-4dbc-a265-412d30de204a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:29.691753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f151a-956c-4dbc-a265-412d30de204a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:29.693427 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f151a-956c-4dbc-a265-412d30de204a None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:29.693427 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f151a-956c-4dbc-a265-412d30de204a None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:29.693427 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f151a-956c-4dbc-a265-412d30de204a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:29.694645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f151a-956c-4dbc-a265-412d30de204a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:29.715754 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 175/352] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:29.725829 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6105cd-d685-4c6c-a88c-106bb9ee9013 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:29.726638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6105cd-d685-4c6c-a88c-106bb9ee9013 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:29.727277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6105cd-d685-4c6c-a88c-106bb9ee9013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:29.760481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2d6105cd-d685-4c6c-a88c-106bb9ee9013 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:29.763007 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2d6105cd-d685-4c6c-a88c-106bb9ee9013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:29.783582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70221d38-b44c-43fc-a42d-7095daac5721 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:29.842398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcded230-e271-44c0-b3d9-a77b67bed65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:29.842826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcded230-e271-44c0-b3d9-a77b67bed65a None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:29.843140 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcded230-e271-44c0-b3d9-a77b67bed65a None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:29.843412 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcded230-e271-44c0-b3d9-a77b67bed65a None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:29.848452 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcded230-e271-44c0-b3d9-a77b67bed65a None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:29.851339 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcded230-e271-44c0-b3d9-a77b67bed65a None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:29.853721 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 178/353] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:29 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:29.855578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b21c87-eb54-4879-a997-4e3095ca67d2 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:29.855578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b21c87-eb54-4879-a997-4e3095ca67d2 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:29.855578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b21c87-eb54-4879-a997-4e3095ca67d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:29.877551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 176/354] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:29.877794 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-23b21c87-eb54-4879-a997-4e3095ca67d2 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:29.879016 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-23b21c87-eb54-4879-a997-4e3095ca67d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:29.879789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6105cd-d685-4c6c-a88c-106bb9ee9013 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:30.028437 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 179/355] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:30.031549 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b21c87-eb54-4879-a997-4e3095ca67d2 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:30.032366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:30.068696 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6105cd-d685-4c6c-a88c-106bb9ee9013 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:30.077227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:30.127144 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f60bb6-a843-419b-82f4-a07c15ece0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:30.127812 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f60bb6-a843-419b-82f4-a07c15ece0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:30.128229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f60bb6-a843-419b-82f4-a07c15ece0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:30.128506 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f60bb6-a843-419b-82f4-a07c15ece0d5 None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:30.129766 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f60bb6-a843-419b-82f4-a07c15ece0d5 None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:30.131346 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f60bb6-a843-419b-82f4-a07c15ece0d5 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:30.134240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 177/356] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:29 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:30.134817 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f60bb6-a843-419b-82f4-a07c15ece0d5 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:30.264400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b21c87-eb54-4879-a997-4e3095ca67d2 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:30.317425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f60bb6-a843-419b-82f4-a07c15ece0d5 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:30.318116 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bba31-da35-4901-999b-7c4949d0152c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:30.318510 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bba31-da35-4901-999b-7c4949d0152c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:30.318735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bba31-da35-4901-999b-7c4949d0152c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:30.319632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bba31-da35-4901-999b-7c4949d0152c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:30.399247 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c54198-6b77-424c-b365-c7c6e7ed3fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:30.401318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c54198-6b77-424c-b365-c7c6e7ed3fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:30.401318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c54198-6b77-424c-b365-c7c6e7ed3fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:30.401318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c54198-6b77-424c-b365-c7c6e7ed3fa8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:30.422486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c54198-6b77-424c-b365-c7c6e7ed3fa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:30.430100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-07c54198-6b77-424c-b365-c7c6e7ed3fa8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:30.430100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:30.430100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:30.430100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:30.430100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:30.430100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:30.430100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:30.430100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:30.430100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:30.430100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:30.430100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:30.430100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:30.430100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:30.430100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:30.430100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:30.430100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:30.431828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:30.431828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:30.431828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:30.431828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:30.431828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:30.431828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:30.431828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:30.431828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:30.431828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:30.431828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:30.431828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:30.431828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:30.431828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:30.431828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:30.431828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:30.431828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:30.434076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:30.434076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:30.434076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:30.434076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:30.434076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:30.434076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:30.434076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:30.434076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:30.434076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:30.434076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:30.434076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:30.434076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:30.435226 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6bba31-da35-4901-999b-7c4949d0152c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:30.435226 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6bba31-da35-4901-999b-7c4949d0152c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:30.435447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 178/357] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:30.435447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c54198-6b77-424c-b365-c7c6e7ed3fa8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:30.448413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:30.456213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:30.630127 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 180/358] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:30.631265 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bba31-da35-4901-999b-7c4949d0152c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:30.644535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c54198-6b77-424c-b365-c7c6e7ed3fa8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:30.703339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae174e-fa9f-475d-b73c-03bbd085b6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:30.704451 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae174e-fa9f-475d-b73c-03bbd085b6f2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:30.704917 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae174e-fa9f-475d-b73c-03bbd085b6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:30.705353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae174e-fa9f-475d-b73c-03bbd085b6f2 None admin] PATH_INFO: `/v3/groups/86295264882f4e689435387a9f9025f0` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:30.716337 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae174e-fa9f-475d-b73c-03bbd085b6f2 None admin] RBAC: Authorizing `identity:get_group(group_id=86295264882f4e689435387a9f9025f0)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:30.720075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae174e-fa9f-475d-b73c-03bbd085b6f2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:30.720641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 179/359] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Aug 30 14:30:30 2022] GET /identity/v3/groups/86295264882f4e689435387a9f9025f0 => generated 226 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:30.722044 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae174e-fa9f-475d-b73c-03bbd085b6f2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:30.834865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:30.856407 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bba31-da35-4901-999b-7c4949d0152c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:30.896178 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:30.905527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae174e-fa9f-475d-b73c-03bbd085b6f2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:30.928685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c02a11-6872-486a-8883-7f69e67e3309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:30.929314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c02a11-6872-486a-8883-7f69e67e3309 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:30.929701 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c02a11-6872-486a-8883-7f69e67e3309 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:30.930059 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c02a11-6872-486a-8883-7f69e67e3309 None admin] PATH_INFO: `/v3/groups/86295264882f4e689435387a9f9025f0` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:30.933688 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c02a11-6872-486a-8883-7f69e67e3309 None admin] RBAC: Authorizing `identity:get_group(group_id=86295264882f4e689435387a9f9025f0)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:30.934956 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c02a11-6872-486a-8883-7f69e67e3309 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:30.938440 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 181/360] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Aug 30 14:30:30 2022] GET /identity/v3/groups/86295264882f4e689435387a9f9025f0 => generated 226 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:30.939470 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c02a11-6872-486a-8883-7f69e67e3309 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:30.966012 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07132b00-820f-451a-9313-c5aeafe39439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:30.967102 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07132b00-820f-451a-9313-c5aeafe39439 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:30.967594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07132b00-820f-451a-9313-c5aeafe39439 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:30.968129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07132b00-820f-451a-9313-c5aeafe39439 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:30.992727 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07132b00-820f-451a-9313-c5aeafe39439 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:31.001687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-07132b00-820f-451a-9313-c5aeafe39439 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:30:31.001687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:31.001687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:31.001687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:31.001687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:31.001687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:31.001687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:31.001687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:31.001687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:31.001687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:31.001687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:31.001687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:31.001687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:31.001687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:31.001687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:31.001687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:31.003418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:31.003418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:31.003418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:31.003418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:31.003418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:31.003418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:31.003418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:31.003418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:31.003418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:31.003418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:31.003418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:31.003418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:31.003418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:31.003418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:31.003418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:31.003418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:31.005157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:31.005157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:31.005157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:31.005157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:31.005157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:31.005157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:31.005157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:31.005157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:31.005157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:31.005157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:31.005157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:30:31.005157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:31.005157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 180/361] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:31.006669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07132b00-820f-451a-9313-c5aeafe39439 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:31.133074 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c02a11-6872-486a-8883-7f69e67e3309 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:31.158612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07132b00-820f-451a-9313-c5aeafe39439 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:31.184316 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3225d-83f7-4159-aa31-8ffc0d911f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:31.184945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3225d-83f7-4159-aa31-8ffc0d911f55 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:31.185217 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3225d-83f7-4159-aa31-8ffc0d911f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:31.185528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3225d-83f7-4159-aa31-8ffc0d911f55 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:31.186177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3225d-83f7-4159-aa31-8ffc0d911f55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:31.186475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3225d-83f7-4159-aa31-8ffc0d911f55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:31.188264 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3225d-83f7-4159-aa31-8ffc0d911f55 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:31.189512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:31.197860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 182/362] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:31.198727 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3225d-83f7-4159-aa31-8ffc0d911f55 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:31.202214 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abbb81-d8c4-4f5e-9b5e-d5fb8b8ec7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:31.203273 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abbb81-d8c4-4f5e-9b5e-d5fb8b8ec7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:31.203740 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abbb81-d8c4-4f5e-9b5e-d5fb8b8ec7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:31.204242 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abbb81-d8c4-4f5e-9b5e-d5fb8b8ec7f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:31.223743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abbb81-d8c4-4f5e-9b5e-d5fb8b8ec7f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:31.232941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-a0abbb81-d8c4-4f5e-9b5e-d5fb8b8ec7f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:31.232941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:31.232941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:31.232941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:31.232941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:31.232941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:31.232941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:31.232941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:31.232941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:31.232941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:31.232941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:31.232941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:31.232941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:31.232941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:31.232941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:31.232941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:31.234417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:31.234417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:31.234417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:31.234417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:31.234417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:31.234417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:31.234417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:31.234417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:31.234417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:31.234417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:31.234417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:31.234417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:31.234417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:31.234417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:31.234417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:31.234417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:31.235798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:31.235798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:31.235798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:31.235798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:31.235798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:31.235798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:31.235798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:31.235798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:31.235798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:31.235798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:31.235798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:31.235798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:31.235798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 181/363] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:31.237795 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abbb81-d8c4-4f5e-9b5e-d5fb8b8ec7f9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:31.257935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:31.328708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3225d-83f7-4159-aa31-8ffc0d911f55 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:31.377821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abbb81-d8c4-4f5e-9b5e-d5fb8b8ec7f9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:31.382842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95cf39-d8d9-4d37-a0bd-62db6b8bf923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:31.382842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95cf39-d8d9-4d37-a0bd-62db6b8bf923 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:31.382842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95cf39-d8d9-4d37-a0bd-62db6b8bf923 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:31.382842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95cf39-d8d9-4d37-a0bd-62db6b8bf923 None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:31.385075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95cf39-d8d9-4d37-a0bd-62db6b8bf923 None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:31.387538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95cf39-d8d9-4d37-a0bd-62db6b8bf923 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:31.393523 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 183/364] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:31 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:31.393523 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95cf39-d8d9-4d37-a0bd-62db6b8bf923 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:31.423262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ac51e-922f-4d60-9c3b-a4eb9f85dabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:31.423787 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ac51e-922f-4d60-9c3b-a4eb9f85dabd None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:31.424134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ac51e-922f-4d60-9c3b-a4eb9f85dabd None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:31.424399 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ac51e-922f-4d60-9c3b-a4eb9f85dabd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:31.425146 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ac51e-922f-4d60-9c3b-a4eb9f85dabd None admin] RBAC: Adding query filter params (group.id=86295264882f4e689435387a9f9025f0, role.id=4eebd8098ab44bfc84cd3de29a679bdf, scope.project.id=e02f7e20d2404b3888cc2e57180a901b) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:31.425449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ac51e-922f-4d60-9c3b-a4eb9f85dabd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:31.426922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ac51e-922f-4d60-9c3b-a4eb9f85dabd None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:31.432851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 182/365] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Aug 30 14:30:31 2022] GET /identity/v3/role_assignments?group.id=86295264882f4e689435387a9f9025f0&scope.project.id=e02f7e20d2404b3888cc2e57180a901b&role.id=4eebd8098ab44bfc84cd3de29a679bdf => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:31.433686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ac51e-922f-4d60-9c3b-a4eb9f85dabd None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:31.518735 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:31.544763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95cf39-d8d9-4d37-a0bd-62db6b8bf923 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:31.574101 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:31.603955 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ac51e-922f-4d60-9c3b-a4eb9f85dabd None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:31.608729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f3bf5-7892-49d2-b754-e41234451715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:31.609574 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f3bf5-7892-49d2-b754-e41234451715 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:31.609937 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f3bf5-7892-49d2-b754-e41234451715 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:31.610385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f3bf5-7892-49d2-b754-e41234451715 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:31.611452 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f3bf5-7892-49d2-b754-e41234451715 None admin] RBAC: Adding query filter params (group.id=86295264882f4e689435387a9f9025f0, role.id=aedf4e1e5c7848099d7bd183778f5896, scope.project.id=e02f7e20d2404b3888cc2e57180a901b) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:31.611884 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f3bf5-7892-49d2-b754-e41234451715 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:31.613547 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f3bf5-7892-49d2-b754-e41234451715 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:31.621943 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 184/366] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Aug 30 14:30:31 2022] GET /identity/v3/role_assignments?group.id=86295264882f4e689435387a9f9025f0&scope.project.id=e02f7e20d2404b3888cc2e57180a901b&role.id=aedf4e1e5c7848099d7bd183778f5896 => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:31.623639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f3bf5-7892-49d2-b754-e41234451715 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:31.689767 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044067c8-9bb1-40c2-8470-04f9405f697e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:31.692290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044067c8-9bb1-40c2-8470-04f9405f697e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:31.693257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044067c8-9bb1-40c2-8470-04f9405f697e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:31.694753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044067c8-9bb1-40c2-8470-04f9405f697e None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:31.696751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044067c8-9bb1-40c2-8470-04f9405f697e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:31.697934 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044067c8-9bb1-40c2-8470-04f9405f697e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:31.702323 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044067c8-9bb1-40c2-8470-04f9405f697e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:31.724055 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 183/367] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:31.724055 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044067c8-9bb1-40c2-8470-04f9405f697e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:31.828061 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f3bf5-7892-49d2-b754-e41234451715 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:31.847322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044067c8-9bb1-40c2-8470-04f9405f697e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:31.863307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1454f2b-18c2-4e29-af8d-5c60abbc583e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:31.863770 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:31.865912 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1454f2b-18c2-4e29-af8d-5c60abbc583e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:31.865912 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1454f2b-18c2-4e29-af8d-5c60abbc583e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:31.865912 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1454f2b-18c2-4e29-af8d-5c60abbc583e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:31.886861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28dfe8-b453-48e4-8de8-5d18eecebdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:31.887491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28dfe8-b453-48e4-8de8-5d18eecebdad None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:31.887753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28dfe8-b453-48e4-8de8-5d18eecebdad None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:31.892047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28dfe8-b453-48e4-8de8-5d18eecebdad None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:31.892047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da28dfe8-b453-48e4-8de8-5d18eecebdad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:31.892047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da28dfe8-b453-48e4-8de8-5d18eecebdad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:31.892047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da28dfe8-b453-48e4-8de8-5d18eecebdad None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:31.906656 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 184/368] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:31.907674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28dfe8-b453-48e4-8de8-5d18eecebdad None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:31.928394 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:31.943428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1454f2b-18c2-4e29-af8d-5c60abbc583e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:31.944838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1454f2b-18c2-4e29-af8d-5c60abbc583e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:32.056713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 185/369] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:32.062752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1454f2b-18c2-4e29-af8d-5c60abbc583e None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:32.118735 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28dfe8-b453-48e4-8de8-5d18eecebdad None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:32.162709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:32.185144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1454f2b-18c2-4e29-af8d-5c60abbc583e None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:32.185788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9e57b-b6ab-4428-8448-6b0071cdd19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:32.186863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9e57b-b6ab-4428-8448-6b0071cdd19e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:32.187487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9e57b-b6ab-4428-8448-6b0071cdd19e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:32.187964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9e57b-b6ab-4428-8448-6b0071cdd19e None admin] PATH_INFO: `/v3/groups/86295264882f4e689435387a9f9025f0` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:32.190068 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9e57b-b6ab-4428-8448-6b0071cdd19e None admin] RBAC: Authorizing `identity:get_group(group_id=86295264882f4e689435387a9f9025f0)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:32.192799 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9e57b-b6ab-4428-8448-6b0071cdd19e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:32.195898 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 185/370] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Aug 30 14:30:31 2022] GET /identity/v3/groups/86295264882f4e689435387a9f9025f0 => generated 226 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:32.197850 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9e57b-b6ab-4428-8448-6b0071cdd19e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:32.227750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e69229-4ed9-46f8-91c2-51df30a483ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:32.228865 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e69229-4ed9-46f8-91c2-51df30a483ec None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:32.229335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e69229-4ed9-46f8-91c2-51df30a483ec None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:32.229815 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e69229-4ed9-46f8-91c2-51df30a483ec None admin] PATH_INFO: `/v3/users/6016bf7c58b045ddb36aa0efd2ef29fd` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:32.231405 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e69229-4ed9-46f8-91c2-51df30a483ec None admin] RBAC: Authorizing `identity:get_user(user_id=6016bf7c58b045ddb36aa0efd2ef29fd)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:32.233096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e69229-4ed9-46f8-91c2-51df30a483ec None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:32.235165 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 186/371] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:32 2022] GET /identity/v3/users/6016bf7c58b045ddb36aa0efd2ef29fd => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:32.236344 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e69229-4ed9-46f8-91c2-51df30a483ec None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:32.260691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:32.317466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9e57b-b6ab-4428-8448-6b0071cdd19e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:32.354805 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212216d3-5c6b-4fcf-8776-bc920e444efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:32.355435 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212216d3-5c6b-4fcf-8776-bc920e444efb None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:32.355712 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212216d3-5c6b-4fcf-8776-bc920e444efb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:32.355979 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212216d3-5c6b-4fcf-8776-bc920e444efb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:32.357676 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e69229-4ed9-46f8-91c2-51df30a483ec None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:32.403141 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c23aa-41f7-4ef7-a111-752e7427cd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:32.404233 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c23aa-41f7-4ef7-a111-752e7427cd69 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:32.404782 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c23aa-41f7-4ef7-a111-752e7427cd69 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:32.405279 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c23aa-41f7-4ef7-a111-752e7427cd69 None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:32.406808 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c23aa-41f7-4ef7-a111-752e7427cd69 None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:32.408731 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c23aa-41f7-4ef7-a111-752e7427cd69 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:32.415707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 187/372] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:32 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:32.417048 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c23aa-41f7-4ef7-a111-752e7427cd69 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:32.441032 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:32.451975 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212216d3-5c6b-4fcf-8776-bc920e444efb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:32.454328 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212216d3-5c6b-4fcf-8776-bc920e444efb None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:32.511328 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:32.533779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c23aa-41f7-4ef7-a111-752e7427cd69 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:32.575211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 186/373] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:32.578232 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212216d3-5c6b-4fcf-8776-bc920e444efb None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:32.587328 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd620c-32ca-49c0-b6ee-9f3c335fbc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:32.587843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd620c-32ca-49c0-b6ee-9f3c335fbc64 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:32.588159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd620c-32ca-49c0-b6ee-9f3c335fbc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:32.588422 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd620c-32ca-49c0-b6ee-9f3c335fbc64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:32.589181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd620c-32ca-49c0-b6ee-9f3c335fbc64 None admin] RBAC: Adding query filter params (group.id=86295264882f4e689435387a9f9025f0, role.id=4eebd8098ab44bfc84cd3de29a679bdf, scope.project.id=b994c620a91d441aa453e5be1b958864) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:32.589488 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd620c-32ca-49c0-b6ee-9f3c335fbc64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:32.590938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd620c-32ca-49c0-b6ee-9f3c335fbc64 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:32.597505 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 188/374] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Aug 30 14:30:32 2022] GET /identity/v3/role_assignments?group.id=86295264882f4e689435387a9f9025f0&scope.project.id=b994c620a91d441aa453e5be1b958864&role.id=4eebd8098ab44bfc84cd3de29a679bdf => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:32.598186 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd620c-32ca-49c0-b6ee-9f3c335fbc64 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:32.697688 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:32.722130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd620c-32ca-49c0-b6ee-9f3c335fbc64 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:32.731113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212216d3-5c6b-4fcf-8776-bc920e444efb None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:32.762999 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f850878-5eb0-49be-83da-ba645714b8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:32.763957 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f850878-5eb0-49be-83da-ba645714b8eb None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:32.764446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f850878-5eb0-49be-83da-ba645714b8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:32.764872 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f850878-5eb0-49be-83da-ba645714b8eb None admin] PATH_INFO: `/v3/groups/86295264882f4e689435387a9f9025f0` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:32.766777 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f850878-5eb0-49be-83da-ba645714b8eb None admin] RBAC: Authorizing `identity:get_group(group_id=86295264882f4e689435387a9f9025f0)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:32.768518 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f850878-5eb0-49be-83da-ba645714b8eb None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:32.770419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 189/375] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Aug 30 14:30:32 2022] GET /identity/v3/groups/86295264882f4e689435387a9f9025f0 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:32.771496 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f850878-5eb0-49be-83da-ba645714b8eb None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:32.802680 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff1c90-f0f5-4950-b1b2-30d5dddf8c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:32.806305 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff1c90-f0f5-4950-b1b2-30d5dddf8c9a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:32.806857 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff1c90-f0f5-4950-b1b2-30d5dddf8c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:32.807581 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff1c90-f0f5-4950-b1b2-30d5dddf8c9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:32.810405 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff1c90-f0f5-4950-b1b2-30d5dddf8c9a None admin] RBAC: Adding query filter params (role.id=89f1b9db96764371860f845c00419e7a, user.id=6016bf7c58b045ddb36aa0efd2ef29fd, scope.system=all) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:32.811094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff1c90-f0f5-4950-b1b2-30d5dddf8c9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:32.819025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff1c90-f0f5-4950-b1b2-30d5dddf8c9a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:32.844200 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 187/376] 10.222.0.25 () {56 vars in 1308 bytes} [Tue Aug 30 14:30:32 2022] GET /identity/v3/role_assignments?user.id=6016bf7c58b045ddb36aa0efd2ef29fd&scope.system=all&role.id=89f1b9db96764371860f845c00419e7a => generated 510 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:32.845832 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff1c90-f0f5-4950-b1b2-30d5dddf8c9a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:32.855288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:32.952097 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f850878-5eb0-49be-83da-ba645714b8eb None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:33.053516 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbad98-772a-41e0-bd2d-799b40cf90ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:33.054428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbad98-772a-41e0-bd2d-799b40cf90ec None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:33.054867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbad98-772a-41e0-bd2d-799b40cf90ec None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:33.055338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbad98-772a-41e0-bd2d-799b40cf90ec None admin] PATH_INFO: `/v3/projects/f9ccb338f4e0425cad7e44cb87838ce1` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:33.057296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbad98-772a-41e0-bd2d-799b40cf90ec None admin] RBAC: Authorizing `identity:get_project(project_id=f9ccb338f4e0425cad7e44cb87838ce1)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:33.059068 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbad98-772a-41e0-bd2d-799b40cf90ec None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:33.065589 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 190/377] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:32 2022] GET /identity/v3/projects/f9ccb338f4e0425cad7e44cb87838ce1 => generated 344 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:33.069001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbad98-772a-41e0-bd2d-799b40cf90ec None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:33.118511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:33.138058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff1c90-f0f5-4950-b1b2-30d5dddf8c9a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:33.214204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec6039-1d36-419c-93bf-747092d6a173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:33.215234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec6039-1d36-419c-93bf-747092d6a173 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:33.215708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec6039-1d36-419c-93bf-747092d6a173 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:33.216228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec6039-1d36-419c-93bf-747092d6a173 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:33.217198 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec6039-1d36-419c-93bf-747092d6a173 None admin] RBAC: Adding query filter params (group.id=86295264882f4e689435387a9f9025f0, role.id=aedf4e1e5c7848099d7bd183778f5896, scope.project.id=b994c620a91d441aa453e5be1b958864) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:33.217706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec6039-1d36-419c-93bf-747092d6a173 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:33.220167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec6039-1d36-419c-93bf-747092d6a173 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:33.228130 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 188/378] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Aug 30 14:30:32 2022] GET /identity/v3/role_assignments?group.id=86295264882f4e689435387a9f9025f0&scope.project.id=b994c620a91d441aa453e5be1b958864&role.id=aedf4e1e5c7848099d7bd183778f5896 => generated 260 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:33.294734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbad98-772a-41e0-bd2d-799b40cf90ec None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:33.359876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6d9c4-1446-4bbc-984f-fdb82980c59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:33.360687 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6d9c4-1446-4bbc-984f-fdb82980c59b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:33.360969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6d9c4-1446-4bbc-984f-fdb82980c59b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:33.361249 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6d9c4-1446-4bbc-984f-fdb82980c59b None admin] PATH_INFO: `/v3/groups/1869d43de3f74a41844e5016f3f6a6a5` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:33.371430 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6d9c4-1446-4bbc-984f-fdb82980c59b None admin] RBAC: Authorizing `identity:get_group(group_id=1869d43de3f74a41844e5016f3f6a6a5)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:33.372977 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6d9c4-1446-4bbc-984f-fdb82980c59b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:33.375208 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 191/379] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Aug 30 14:30:33 2022] GET /identity/v3/groups/1869d43de3f74a41844e5016f3f6a6a5 => generated 229 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:33.382739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec6039-1d36-419c-93bf-747092d6a173 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:33.512407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:33.553966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec6039-1d36-419c-93bf-747092d6a173 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:33.577416 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e8093-c264-4287-bca4-53ca951a4079 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:33.577416 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e8093-c264-4287-bca4-53ca951a4079 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:33.577416 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e8093-c264-4287-bca4-53ca951a4079 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:33.578262 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 192/380] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:33.592472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156020d-c277-4e7f-b58a-516c136e5a2b None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:33.592790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156020d-c277-4e7f-b58a-516c136e5a2b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:33.593027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156020d-c277-4e7f-b58a-516c136e5a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:33.602784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf06c5-6507-441e-b7a4-85c46f260276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:33.603835 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf06c5-6507-441e-b7a4-85c46f260276 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:33.604490 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf06c5-6507-441e-b7a4-85c46f260276 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:33.604970 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf06c5-6507-441e-b7a4-85c46f260276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:33.605940 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf06c5-6507-441e-b7a4-85c46f260276 None admin] RBAC: Adding query filter params (group.id=1869d43de3f74a41844e5016f3f6a6a5, role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=f9ccb338f4e0425cad7e44cb87838ce1) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:33.606479 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf06c5-6507-441e-b7a4-85c46f260276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:33.608221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf06c5-6507-441e-b7a4-85c46f260276 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:33.620040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 189/381] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Aug 30 14:30:33 2022] GET /identity/v3/role_assignments?group.id=1869d43de3f74a41844e5016f3f6a6a5&scope.project.id=f9ccb338f4e0425cad7e44cb87838ce1&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 260 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:33.643080 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a156020d-c277-4e7f-b58a-516c136e5a2b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:33.643080 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a156020d-c277-4e7f-b58a-516c136e5a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:33.815337 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 193/382] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:30:34.102679 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762502d5-9a20-43b6-ae86-5850ff56eb54 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:34.103169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762502d5-9a20-43b6-ae86-5850ff56eb54 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:34.103894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762502d5-9a20-43b6-ae86-5850ff56eb54 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:34.106280 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 190/383] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:34.113054 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01ad2-8224-4a52-b54d-550b14498e52 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:34.113331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01ad2-8224-4a52-b54d-550b14498e52 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:34.113605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01ad2-8224-4a52-b54d-550b14498e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:34.141833 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-2ea01ad2-8224-4a52-b54d-550b14498e52 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:34.143073 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2ea01ad2-8224-4a52-b54d-550b14498e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:34.150265 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:34.289739 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 194/384] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:34.316525 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762502d5-9a20-43b6-ae86-5850ff56eb54 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:34.484080 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762502d5-9a20-43b6-ae86-5850ff56eb54 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:34.550202 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:34.564787 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703b605-9b25-49ea-a168-ef619aa1c69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:34.565466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703b605-9b25-49ea-a168-ef619aa1c69d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:34.565747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703b605-9b25-49ea-a168-ef619aa1c69d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:34.566017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703b605-9b25-49ea-a168-ef619aa1c69d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:34.586929 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5703b605-9b25-49ea-a168-ef619aa1c69d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:34.591991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-5703b605-9b25-49ea-a168-ef619aa1c69d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:34.591991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:34.591991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:34.591991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:34.591991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:34.591991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:34.591991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:34.591991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:34.591991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:34.591991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:34.591991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:34.591991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:34.591991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:34.591991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:34.591991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:34.591991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:34.592933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:34.592933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:34.592933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:34.592933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:34.592933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:34.592933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:34.592933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:34.592933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:34.592933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:34.592933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:34.592933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:34.592933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:34.592933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:34.592933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:34.592933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:34.592933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:34.593786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:34.593786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:34.593786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:34.593786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:34.593786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:34.593786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:34.593786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:34.593786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:34.593786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:34.593786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:34.593786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:34.593786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:34.601764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 191/385] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:34.609219 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea01ad2-8224-4a52-b54d-550b14498e52 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:34.735300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703b605-9b25-49ea-a168-ef619aa1c69d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:34.899214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea01ad2-8224-4a52-b54d-550b14498e52 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:34.970648 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703b605-9b25-49ea-a168-ef619aa1c69d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:35.018584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf03ec9-709b-48c7-9954-46c1d7e8e887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:35.019149 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf03ec9-709b-48c7-9954-46c1d7e8e887 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:35.019432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf03ec9-709b-48c7-9954-46c1d7e8e887 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:35.019704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf03ec9-709b-48c7-9954-46c1d7e8e887 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:35.020434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf03ec9-709b-48c7-9954-46c1d7e8e887 None admin] RBAC: Adding query filter params (name=member) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:35.020737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf03ec9-709b-48c7-9954-46c1d7e8e887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:35.022153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf03ec9-709b-48c7-9954-46c1d7e8e887 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:35.064709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 195/386] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 456 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:35.083738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf03ec9-709b-48c7-9954-46c1d7e8e887 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:35.087462 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:35.098513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b913f4c-fb91-40a0-93da-0335891b913d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:35.099535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b913f4c-fb91-40a0-93da-0335891b913d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:35.100021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b913f4c-fb91-40a0-93da-0335891b913d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:35.111082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b913f4c-fb91-40a0-93da-0335891b913d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:35.158363 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b913f4c-fb91-40a0-93da-0335891b913d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:35.178512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-1b913f4c-fb91-40a0-93da-0335891b913d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:35.178512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:35.178512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:35.178512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:35.178512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:35.178512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:35.178512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:35.178512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:35.178512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:35.178512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:35.178512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:35.178512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:35.178512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:35.178512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:35.178512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:35.178512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:35.179487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:35.179487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:35.179487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:35.179487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:35.179487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:35.179487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:35.179487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:35.179487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:35.179487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:35.179487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:35.179487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:35.179487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:35.179487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:35.179487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:35.179487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:35.179487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:35.180336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:35.180336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:35.180336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:35.180336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:35.180336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:35.180336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:35.180336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:35.180336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:35.180336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:35.180336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:35.180336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:35.180336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:35.184560 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 192/387] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 448 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:35.191763 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b913f4c-fb91-40a0-93da-0335891b913d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:35.352425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:35.537748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf03ec9-709b-48c7-9954-46c1d7e8e887 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:35.541943 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b913f4c-fb91-40a0-93da-0335891b913d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:35.668706 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4fbdd-55ff-47ee-9b3b-c85b28f72a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:35.669381 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4fbdd-55ff-47ee-9b3b-c85b28f72a2f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:35.672531 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4fbdd-55ff-47ee-9b3b-c85b28f72a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:35.672531 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4fbdd-55ff-47ee-9b3b-c85b28f72a2f None admin] PATH_INFO: `/v3/groups/86295264882f4e689435387a9f9025f0` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:35.672782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63c3c7-f083-4e11-8d00-d17f4816bbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:35.672782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63c3c7-f083-4e11-8d00-d17f4816bbac None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:35.673247 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63c3c7-f083-4e11-8d00-d17f4816bbac None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:35.673247 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63c3c7-f083-4e11-8d00-d17f4816bbac None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:35.673922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63c3c7-f083-4e11-8d00-d17f4816bbac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:35.674226 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63c3c7-f083-4e11-8d00-d17f4816bbac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:35.675687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63c3c7-f083-4e11-8d00-d17f4816bbac None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:35.678359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4fbdd-55ff-47ee-9b3b-c85b28f72a2f None admin] RBAC: Authorizing `identity:get_group(group_id=86295264882f4e689435387a9f9025f0)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:35.681508 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4fbdd-55ff-47ee-9b3b-c85b28f72a2f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:35.681984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 196/388] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Aug 30 14:30:35 2022] GET /identity/v3/groups/86295264882f4e689435387a9f9025f0 => generated 226 bytes in 599 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:35.690087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4fbdd-55ff-47ee-9b3b-c85b28f72a2f None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:35.698287 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 193/389] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 505 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:35.709115 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63c3c7-f083-4e11-8d00-d17f4816bbac None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:35.802298 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:35.840657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4fbdd-55ff-47ee-9b3b-c85b28f72a2f None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:35.871940 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63c3c7-f083-4e11-8d00-d17f4816bbac None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:35.885504 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66103e94-ad72-4f21-aeda-8ad13bacafdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:35.886086 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66103e94-ad72-4f21-aeda-8ad13bacafdc None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:35.888032 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66103e94-ad72-4f21-aeda-8ad13bacafdc None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:35.888032 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66103e94-ad72-4f21-aeda-8ad13bacafdc None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:35.888032 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66103e94-ad72-4f21-aeda-8ad13bacafdc None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:35.891740 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66103e94-ad72-4f21-aeda-8ad13bacafdc None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:35.894243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 197/390] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:35 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:35.904025 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57fb75-45c9-41d6-b266-aafff68b3cd7 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:35.904025 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57fb75-45c9-41d6-b266-aafff68b3cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:35.904025 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57fb75-45c9-41d6-b266-aafff68b3cd7 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:35.911353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:35.916026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 198/391] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:35 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:35.916026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57fb75-45c9-41d6-b266-aafff68b3cd7 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:35.958947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff2bed-307a-4bdd-bc92-19d5a350862e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:35.959604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff2bed-307a-4bdd-bc92-19d5a350862e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:35.963345 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff2bed-307a-4bdd-bc92-19d5a350862e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:35.963642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff2bed-307a-4bdd-bc92-19d5a350862e None admin] PATH_INFO: `/v3/groups/86295264882f4e689435387a9f9025f0` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:35.964954 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff2bed-307a-4bdd-bc92-19d5a350862e None admin] RBAC: Authorizing `identity:get_group(group_id=86295264882f4e689435387a9f9025f0)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:35.966446 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff2bed-307a-4bdd-bc92-19d5a350862e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:35.973368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 194/392] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Aug 30 14:30:35 2022] GET /identity/v3/groups/86295264882f4e689435387a9f9025f0 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:35.974775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef1796-5a22-48bd-8301-a205dee6ebff None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:35.975053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef1796-5a22-48bd-8301-a205dee6ebff None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:35.978370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef1796-5a22-48bd-8301-a205dee6ebff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:36.023067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e8ef1796-5a22-48bd-8301-a205dee6ebff None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:36.025683 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e8ef1796-5a22-48bd-8301-a205dee6ebff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:36.157524 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57fb75-45c9-41d6-b266-aafff68b3cd7 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:36.180934 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 195/393] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:36.181824 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef1796-5a22-48bd-8301-a205dee6ebff None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:36.236831 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60d3e6-f0ec-4646-be50-b285449748b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:36.237529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60d3e6-f0ec-4646-be50-b285449748b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:36.240138 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60d3e6-f0ec-4646-be50-b285449748b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:36.240138 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60d3e6-f0ec-4646-be50-b285449748b5 None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b/groups/86295264882f4e689435387a9f9025f0/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:36.242620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60d3e6-f0ec-4646-be50-b285449748b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e02f7e20d2404b3888cc2e57180a901b, group_id=86295264882f4e689435387a9f9025f0, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:36.244114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60d3e6-f0ec-4646-be50-b285449748b5 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:36.279340 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 199/394] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Aug 30 14:30:35 2022] PUT /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b/groups/86295264882f4e689435387a9f9025f0/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:30:36.279666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:36.387579 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef1796-5a22-48bd-8301-a205dee6ebff None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:36.412418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:36.425007 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798b04b-d625-487b-b306-bdc8fc38abfd None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:36.425965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798b04b-d625-487b-b306-bdc8fc38abfd None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:36.426748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798b04b-d625-487b-b306-bdc8fc38abfd None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:36.440906 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 200/395] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:36 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:30:36.443572 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd301e2-040c-4583-bce5-378e5a2b66f9 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:36.450692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5febc-c215-4fc0-9bd4-12d6d1dda597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:36.451388 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5febc-c215-4fc0-9bd4-12d6d1dda597 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:36.451666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5febc-c215-4fc0-9bd4-12d6d1dda597 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:36.451926 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5febc-c215-4fc0-9bd4-12d6d1dda597 None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:36.452909 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd301e2-040c-4583-bce5-378e5a2b66f9 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:36.453607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5febc-c215-4fc0-9bd4-12d6d1dda597 None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:36.453984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd301e2-040c-4583-bce5-378e5a2b66f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:36.455419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5febc-c215-4fc0-9bd4-12d6d1dda597 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:36.457637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 196/396] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:36 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:36.468149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5febc-c215-4fc0-9bd4-12d6d1dda597 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:36.528399 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-3fd301e2-040c-4583-bce5-378e5a2b66f9 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:36.530104 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-3fd301e2-040c-4583-bce5-378e5a2b66f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:36.651202 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5febc-c215-4fc0-9bd4-12d6d1dda597 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:36.728105 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 201/397] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:36.736812 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f163d2-d57f-4ab1-8eb5-5179bf3adc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:36.738192 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f163d2-d57f-4ab1-8eb5-5179bf3adc66 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:36.738790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f163d2-d57f-4ab1-8eb5-5179bf3adc66 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:36.739320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f163d2-d57f-4ab1-8eb5-5179bf3adc66 None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b/groups/86295264882f4e689435387a9f9025f0/roles/aedf4e1e5c7848099d7bd183778f5896` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:36.750173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f163d2-d57f-4ab1-8eb5-5179bf3adc66 None admin] RBAC: Authorizing `identity:create_grant(project_id=e02f7e20d2404b3888cc2e57180a901b, group_id=86295264882f4e689435387a9f9025f0, role_id=aedf4e1e5c7848099d7bd183778f5896)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:36.752221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f163d2-d57f-4ab1-8eb5-5179bf3adc66 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:36.782564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 197/398] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Aug 30 14:30:36 2022] PUT /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b/groups/86295264882f4e689435387a9f9025f0/roles/aedf4e1e5c7848099d7bd183778f5896 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:30:36.805267 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd301e2-040c-4583-bce5-378e5a2b66f9 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:36.861459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fc74f-4136-45b7-90ac-db2ea5448c56 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:36.862959 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fc74f-4136-45b7-90ac-db2ea5448c56 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:36.863493 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fc74f-4136-45b7-90ac-db2ea5448c56 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:36.865942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 198/399] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:36.873656 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481a09e-54b6-4294-8a66-ffadf867d55c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:36.874298 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481a09e-54b6-4294-8a66-ffadf867d55c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:36.874831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481a09e-54b6-4294-8a66-ffadf867d55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:36.919122 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7481a09e-54b6-4294-8a66-ffadf867d55c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:36.920869 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7481a09e-54b6-4294-8a66-ffadf867d55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:37.018906 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:37.097950 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd301e2-040c-4583-bce5-378e5a2b66f9 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:37.120969 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 199/400] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:30:37.152610 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:37.178327 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83314367-f816-43de-a48d-54fcc2ac1bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:37.179790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83314367-f816-43de-a48d-54fcc2ac1bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:37.180578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83314367-f816-43de-a48d-54fcc2ac1bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:37.181305 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83314367-f816-43de-a48d-54fcc2ac1bb8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:37.211191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83314367-f816-43de-a48d-54fcc2ac1bb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:37.224177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-83314367-f816-43de-a48d-54fcc2ac1bb8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:37.224177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:37.224177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:37.224177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:37.224177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:37.224177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:37.224177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:37.224177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:37.224177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:37.224177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:37.224177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:37.224177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:37.224177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:37.224177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:37.224177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:37.224177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:37.226174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:37.226174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:37.226174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:37.226174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:37.226174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:37.226174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:37.226174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:37.226174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:37.226174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:37.226174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:37.226174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:37.226174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:37.226174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:37.226174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:37.226174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:37.226174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:37.227703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:37.227703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:37.227703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:37.227703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:37.227703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:37.227703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:37.227703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:37.227703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:37.227703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:37.227703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:37.227703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:37.227703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:37.236183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 202/401] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:37.242998 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481a09e-54b6-4294-8a66-ffadf867d55c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:37.306163 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83314367-f816-43de-a48d-54fcc2ac1bb8 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:37.535320 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83314367-f816-43de-a48d-54fcc2ac1bb8 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:37.596526 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481a09e-54b6-4294-8a66-ffadf867d55c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:37.598785 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30ca89-4f81-452e-9dbe-c6f11afca640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:37.599687 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30ca89-4f81-452e-9dbe-c6f11afca640 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:37.599966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30ca89-4f81-452e-9dbe-c6f11afca640 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:37.602578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30ca89-4f81-452e-9dbe-c6f11afca640 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:37.616693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd30ca89-4f81-452e-9dbe-c6f11afca640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:37.622538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-dd30ca89-4f81-452e-9dbe-c6f11afca640 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:37.622538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:37.622538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:37.622538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:37.622538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:37.622538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:37.622538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:37.622538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:37.622538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:37.622538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:37.622538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:37.622538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:37.622538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:37.622538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:37.622538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:37.622538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:37.623526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:37.623526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:37.623526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:37.623526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:37.623526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:37.623526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:37.623526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:37.623526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:37.623526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:37.623526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:37.623526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:37.623526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:37.623526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:37.623526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:37.623526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:37.623526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:37.624386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:37.624386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:37.624386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:37.624386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:37.624386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:37.624386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:37.624386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:37.624386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:37.624386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:37.624386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:37.624386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:37.624386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:37.628184 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 203/402] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:37.629349 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30ca89-4f81-452e-9dbe-c6f11afca640 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:37.689068 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff280ae6-4a82-4719-84b0-6c8bd92e8515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:37.690530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff280ae6-4a82-4719-84b0-6c8bd92e8515 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:37.694151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff280ae6-4a82-4719-84b0-6c8bd92e8515 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:37.694151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff280ae6-4a82-4719-84b0-6c8bd92e8515 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:37.694151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff280ae6-4a82-4719-84b0-6c8bd92e8515 None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:37.694151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff280ae6-4a82-4719-84b0-6c8bd92e8515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:37.694151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff280ae6-4a82-4719-84b0-6c8bd92e8515 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:37.709892 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 200/403] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 468 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:37.710922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff280ae6-4a82-4719-84b0-6c8bd92e8515 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:37.736127 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:37.755298 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:37.847750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30ca89-4f81-452e-9dbe-c6f11afca640 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:37.850423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff280ae6-4a82-4719-84b0-6c8bd92e8515 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:37.891900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5d0a4-82c7-49af-b596-adf7d899e63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:37.892876 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5d0a4-82c7-49af-b596-adf7d899e63b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:37.893705 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5d0a4-82c7-49af-b596-adf7d899e63b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:37.894266 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5d0a4-82c7-49af-b596-adf7d899e63b None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:37.897228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5d0a4-82c7-49af-b596-adf7d899e63b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:37.898177 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5d0a4-82c7-49af-b596-adf7d899e63b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:37.901286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5d0a4-82c7-49af-b596-adf7d899e63b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:37.921407 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02704132-eaf0-4ac6-b7f1-eab96dca96f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:37.923327 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02704132-eaf0-4ac6-b7f1-eab96dca96f5 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:37.923752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02704132-eaf0-4ac6-b7f1-eab96dca96f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:37.924057 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02704132-eaf0-4ac6-b7f1-eab96dca96f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:37.934695 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 201/404] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:37.936147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5d0a4-82c7-49af-b596-adf7d899e63b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:37.951729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02704132-eaf0-4ac6-b7f1-eab96dca96f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:37.966296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-02704132-eaf0-4ac6-b7f1-eab96dca96f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:37.966296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:37.966296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:37.966296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:37.966296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:37.966296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:37.966296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:37.966296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:37.966296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:37.966296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:37.966296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:37.966296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:37.966296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:37.966296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:37.966296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:37.966296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:37.967665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:37.967665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:37.967665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:37.967665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:37.967665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:37.967665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:37.967665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:37.967665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:37.967665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:37.967665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:37.967665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:37.967665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:37.967665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:37.967665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:37.967665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:37.967665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:37.968578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:37.968578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:37.968578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:37.968578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:37.968578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:37.968578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:37.968578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:37.968578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:37.968578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:37.968578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:37.968578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:37.968578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:37.968578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 204/405] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:37.969643 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02704132-eaf0-4ac6-b7f1-eab96dca96f5 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:38.126651 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:38.146479 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5d0a4-82c7-49af-b596-adf7d899e63b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:38.190985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02704132-eaf0-4ac6-b7f1-eab96dca96f5 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:38.210584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:38.227756 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6d96c-876a-43cc-b196-3edeb726160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:38.228459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6d96c-876a-43cc-b196-3edeb726160c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:38.228731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6d96c-876a-43cc-b196-3edeb726160c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:38.229003 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6d96c-876a-43cc-b196-3edeb726160c None admin] PATH_INFO: `/v3/groups/86295264882f4e689435387a9f9025f0` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:38.232215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6d96c-876a-43cc-b196-3edeb726160c None admin] RBAC: Authorizing `identity:get_group(group_id=86295264882f4e689435387a9f9025f0)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:38.233821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6d96c-876a-43cc-b196-3edeb726160c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:38.235707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 202/406] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Aug 30 14:30:37 2022] GET /identity/v3/groups/86295264882f4e689435387a9f9025f0 => generated 226 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:38.236638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6d96c-876a-43cc-b196-3edeb726160c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:38.272386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52495929-f5c5-4baa-84bb-5c04ab2efde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:38.273049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495929-f5c5-4baa-84bb-5c04ab2efde3 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:38.273328 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495929-f5c5-4baa-84bb-5c04ab2efde3 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:38.273591 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495929-f5c5-4baa-84bb-5c04ab2efde3 None admin] PATH_INFO: `/v3/groups/86295264882f4e689435387a9f9025f0` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:38.275837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52495929-f5c5-4baa-84bb-5c04ab2efde3 None admin] RBAC: Authorizing `identity:get_group(group_id=86295264882f4e689435387a9f9025f0)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:38.277321 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52495929-f5c5-4baa-84bb-5c04ab2efde3 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:38.278988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 205/407] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Aug 30 14:30:37 2022] GET /identity/v3/groups/86295264882f4e689435387a9f9025f0 => generated 226 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:38.285489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52495929-f5c5-4baa-84bb-5c04ab2efde3 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:38.526796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6d96c-876a-43cc-b196-3edeb726160c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:38.605331 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b177d-f5de-419c-8293-9532e46d2354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:38.605975 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b177d-f5de-419c-8293-9532e46d2354 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:38.606245 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b177d-f5de-419c-8293-9532e46d2354 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:38.608938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52495929-f5c5-4baa-84bb-5c04ab2efde3 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:38.609942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b177d-f5de-419c-8293-9532e46d2354 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:38.611083 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b177d-f5de-419c-8293-9532e46d2354 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:38.613827 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b177d-f5de-419c-8293-9532e46d2354 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:38.615506 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b177d-f5de-419c-8293-9532e46d2354 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:38.641323 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 203/408] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:38.644737 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b177d-f5de-419c-8293-9532e46d2354 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:38.707519 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc8287-d276-4b0f-8016-4d0b6e9aef13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:38.710897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc8287-d276-4b0f-8016-4d0b6e9aef13 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:38.711374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc8287-d276-4b0f-8016-4d0b6e9aef13 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:38.711781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc8287-d276-4b0f-8016-4d0b6e9aef13 None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:38.713072 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:38.715730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc8287-d276-4b0f-8016-4d0b6e9aef13 None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:38.717325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc8287-d276-4b0f-8016-4d0b6e9aef13 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:38.727353 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 206/409] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:38 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:38.729401 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc8287-d276-4b0f-8016-4d0b6e9aef13 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:38.822948 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:38.856161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b177d-f5de-419c-8293-9532e46d2354 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:38.899182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c717ff9-5cdb-4b9b-92fd-f8bd91cb6ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:38.899852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c717ff9-5cdb-4b9b-92fd-f8bd91cb6ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:38.900210 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c717ff9-5cdb-4b9b-92fd-f8bd91cb6ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:38.900482 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c717ff9-5cdb-4b9b-92fd-f8bd91cb6ef6 None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:38.901833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c717ff9-5cdb-4b9b-92fd-f8bd91cb6ef6 None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:38.903227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c717ff9-5cdb-4b9b-92fd-f8bd91cb6ef6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:38.905718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 204/410] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:38 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:38.906381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c717ff9-5cdb-4b9b-92fd-f8bd91cb6ef6 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:38.944450 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc8287-d276-4b0f-8016-4d0b6e9aef13 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:39.020766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81c5cc-1fb9-43ad-93f6-859fc5fb58ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:39.021913 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81c5cc-1fb9-43ad-93f6-859fc5fb58ba None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:39.022395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81c5cc-1fb9-43ad-93f6-859fc5fb58ba None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:39.022872 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81c5cc-1fb9-43ad-93f6-859fc5fb58ba None admin] PATH_INFO: `/v3/groups/1869d43de3f74a41844e5016f3f6a6a5` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:39.025570 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81c5cc-1fb9-43ad-93f6-859fc5fb58ba None admin] RBAC: Authorizing `identity:get_group(group_id=1869d43de3f74a41844e5016f3f6a6a5)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:39.028500 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81c5cc-1fb9-43ad-93f6-859fc5fb58ba None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:39.038634 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 207/411] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Aug 30 14:30:38 2022] GET /identity/v3/groups/1869d43de3f74a41844e5016f3f6a6a5 => generated 229 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:39.038634 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02cbfe-50b2-4fc0-a0a7-71401dbc84fe None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:39.038634 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02cbfe-50b2-4fc0-a0a7-71401dbc84fe None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:39.038634 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02cbfe-50b2-4fc0-a0a7-71401dbc84fe None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:39.039015 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 208/412] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:39.048795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02cbfe-50b2-4fc0-a0a7-71401dbc84fe None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:39.049952 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:39.064585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c717ff9-5cdb-4b9b-92fd-f8bd91cb6ef6 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:39.109932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb04ce-f34e-4db5-bfa1-e3ab274b7af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:39.110542 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb04ce-f34e-4db5-bfa1-e3ab274b7af2 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:39.110819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb04ce-f34e-4db5-bfa1-e3ab274b7af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:39.111087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb04ce-f34e-4db5-bfa1-e3ab274b7af2 None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864/groups/86295264882f4e689435387a9f9025f0/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:39.113400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb04ce-f34e-4db5-bfa1-e3ab274b7af2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b994c620a91d441aa453e5be1b958864, group_id=86295264882f4e689435387a9f9025f0, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:39.115049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb04ce-f34e-4db5-bfa1-e3ab274b7af2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:39.140449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 205/413] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Aug 30 14:30:38 2022] PUT /identity/v3/projects/b994c620a91d441aa453e5be1b958864/groups/86295264882f4e689435387a9f9025f0/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:30:39.141570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb04ce-f34e-4db5-bfa1-e3ab274b7af2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:39.195509 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02cbfe-50b2-4fc0-a0a7-71401dbc84fe None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:39.215640 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:39.266506 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe825ad-caef-4a4a-8e73-5dd28a7a007d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:39.268044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe825ad-caef-4a4a-8e73-5dd28a7a007d None admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:39.268044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe825ad-caef-4a4a-8e73-5dd28a7a007d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:39.268044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe825ad-caef-4a4a-8e73-5dd28a7a007d None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864/groups/86295264882f4e689435387a9f9025f0/roles/aedf4e1e5c7848099d7bd183778f5896` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:39.273585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe825ad-caef-4a4a-8e73-5dd28a7a007d None admin] RBAC: Authorizing `identity:create_grant(project_id=b994c620a91d441aa453e5be1b958864, group_id=86295264882f4e689435387a9f9025f0, role_id=aedf4e1e5c7848099d7bd183778f5896)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:39.276557 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe825ad-caef-4a4a-8e73-5dd28a7a007d None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:39.312769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 209/414] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Aug 30 14:30:39 2022] PUT /identity/v3/projects/b994c620a91d441aa453e5be1b958864/groups/86295264882f4e689435387a9f9025f0/roles/aedf4e1e5c7848099d7bd183778f5896 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:30:39.318691 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1c1f0-f670-4d7f-85fa-b34e7139f458 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:39.319444 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1c1f0-f670-4d7f-85fa-b34e7139f458 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:39.319921 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1c1f0-f670-4d7f-85fa-b34e7139f458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:39.372779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-bfb1c1f0-f670-4d7f-85fa-b34e7139f458 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:39.375120 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-bfb1c1f0-f670-4d7f-85fa-b34e7139f458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:39.484384 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb04ce-f34e-4db5-bfa1-e3ab274b7af2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:39.564033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13c4d6-6d81-4c39-a28d-f008b382f20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:39.564033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13c4d6-6d81-4c39-a28d-f008b382f20f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:39.564033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13c4d6-6d81-4c39-a28d-f008b382f20f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:39.564033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13c4d6-6d81-4c39-a28d-f008b382f20f None admin] PATH_INFO: `/v3/projects/f9ccb338f4e0425cad7e44cb87838ce1` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:39.564452 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13c4d6-6d81-4c39-a28d-f008b382f20f None admin] RBAC: Authorizing `identity:get_project(project_id=f9ccb338f4e0425cad7e44cb87838ce1)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:39.565263 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13c4d6-6d81-4c39-a28d-f008b382f20f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:39.580036 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 206/415] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:39 2022] GET /identity/v3/projects/f9ccb338f4e0425cad7e44cb87838ce1 => generated 344 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:39.584347 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13c4d6-6d81-4c39-a28d-f008b382f20f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:39.622697 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 210/416] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:39.680863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:39.878323 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13c4d6-6d81-4c39-a28d-f008b382f20f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:39.977014 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c64c62-2c52-493b-914a-17ea43cf9607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:39.978109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c64c62-2c52-493b-914a-17ea43cf9607 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:39.978588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c64c62-2c52-493b-914a-17ea43cf9607 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:39.979085 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c64c62-2c52-493b-914a-17ea43cf9607 None admin] PATH_INFO: `/v3/projects/f9ccb338f4e0425cad7e44cb87838ce1/groups/1869d43de3f74a41844e5016f3f6a6a5/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:39.989381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c64c62-2c52-493b-914a-17ea43cf9607 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9ccb338f4e0425cad7e44cb87838ce1, group_id=1869d43de3f74a41844e5016f3f6a6a5, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:39.991223 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c64c62-2c52-493b-914a-17ea43cf9607 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:40.021840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 207/417] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Aug 30 14:30:39 2022] PUT /identity/v3/projects/f9ccb338f4e0425cad7e44cb87838ce1/groups/1869d43de3f74a41844e5016f3f6a6a5/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:30:40.124764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a896a0c-b03f-4209-b23a-ccb3b1c57a90 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:40.125583 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a896a0c-b03f-4209-b23a-ccb3b1c57a90 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:40.125671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a896a0c-b03f-4209-b23a-ccb3b1c57a90 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:40.128931 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 211/418] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:40.130834 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c64c62-2c52-493b-914a-17ea43cf9607 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:40.148330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01964d5-81c3-486c-a377-5c84e0318809 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:40.149579 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01964d5-81c3-486c-a377-5c84e0318809 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:40.150296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01964d5-81c3-486c-a377-5c84e0318809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:40.193745 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a01964d5-81c3-486c-a377-5c84e0318809 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:40.199006 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a01964d5-81c3-486c-a377-5c84e0318809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:40.365616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 212/419] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:40.439875 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:40.527727 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c64c62-2c52-493b-914a-17ea43cf9607 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:40.682743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7be59-1888-467a-b7b0-17b56c9993f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:40.688037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7be59-1888-467a-b7b0-17b56c9993f9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:40.688037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7be59-1888-467a-b7b0-17b56c9993f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:40.688037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7be59-1888-467a-b7b0-17b56c9993f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:40.720714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7be59-1888-467a-b7b0-17b56c9993f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:40.731299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-46a7be59-1888-467a-b7b0-17b56c9993f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:40.731299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:40.731299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:40.731299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:40.731299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:40.731299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:40.731299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:40.731299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:40.731299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:40.731299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:40.731299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:40.731299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:40.731299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:40.731299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:40.731299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:40.731299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:40.733053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:40.733053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:40.733053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:40.733053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:40.733053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:40.733053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:40.733053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:40.733053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:40.733053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:40.733053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:40.733053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:40.733053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:40.733053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:40.733053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:40.733053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:40.733053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:40.734671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:40.734671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:40.734671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:40.734671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:40.734671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:40.734671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:40.734671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:40.734671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:40.734671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:40.734671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:40.734671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:40.734671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:40.742051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 208/420] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 612 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:40.750007 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01964d5-81c3-486c-a377-5c84e0318809 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:40.865474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:41.099173 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01964d5-81c3-486c-a377-5c84e0318809 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:41.185822 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7be59-1888-467a-b7b0-17b56c9993f9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:41.204330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c04641-b739-4a70-ac16-c1e9c689e155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:41.205144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c04641-b739-4a70-ac16-c1e9c689e155 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:41.205267 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c04641-b739-4a70-ac16-c1e9c689e155 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:41.205613 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c04641-b739-4a70-ac16-c1e9c689e155 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:41.206195 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c04641-b739-4a70-ac16-c1e9c689e155 None admin] RBAC: Adding query filter params (name=member) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:41.215585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c04641-b739-4a70-ac16-c1e9c689e155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:41.216841 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c04641-b739-4a70-ac16-c1e9c689e155 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:41.250504 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 213/421] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 503 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:41.262625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c04641-b739-4a70-ac16-c1e9c689e155 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:41.535425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:41.541647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c04641-b739-4a70-ac16-c1e9c689e155 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:41.597241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7be59-1888-467a-b7b0-17b56c9993f9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:41.664027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd86437-0d2a-46cc-b572-c5fd585d0439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:41.665099 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd86437-0d2a-46cc-b572-c5fd585d0439 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:41.665596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd86437-0d2a-46cc-b572-c5fd585d0439 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:41.666082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd86437-0d2a-46cc-b572-c5fd585d0439 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:41.753542 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa23d55-93c4-4f2d-9fc2-5f83fad97787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:41.754988 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa23d55-93c4-4f2d-9fc2-5f83fad97787 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:41.755483 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa23d55-93c4-4f2d-9fc2-5f83fad97787 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:41.757866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa23d55-93c4-4f2d-9fc2-5f83fad97787 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:41.796425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:41.799913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa23d55-93c4-4f2d-9fc2-5f83fad97787 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:41.814930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-9fa23d55-93c4-4f2d-9fc2-5f83fad97787 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:41.814930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:41.814930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:41.814930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:41.814930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:41.814930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:41.814930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:41.814930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:41.814930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:41.814930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:41.814930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:41.814930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:41.814930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:41.814930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:41.814930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:41.814930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:41.816676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:41.816676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:41.816676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:41.816676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:41.816676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:41.816676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:41.816676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:41.816676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:41.816676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:41.816676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:41.816676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:41.816676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:41.816676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:41.816676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:41.816676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:41.816676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:41.818227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:41.818227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:41.818227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:41.818227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:41.818227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:41.818227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:41.818227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:41.818227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:41.818227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:41.818227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:41.818227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:41.818227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:41.818227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 209/422] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 633 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:41.831608 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa23d55-93c4-4f2d-9fc2-5f83fad97787 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:41.916037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd86437-0d2a-46cc-b572-c5fd585d0439 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:41.917958 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd86437-0d2a-46cc-b572-c5fd585d0439 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:42.086806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa23d55-93c4-4f2d-9fc2-5f83fad97787 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:42.157421 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f303f8d-6c24-4152-9912-f962bbfe7f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:42.158690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f303f8d-6c24-4152-9912-f962bbfe7f72 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:42.159295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f303f8d-6c24-4152-9912-f962bbfe7f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:42.164037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f303f8d-6c24-4152-9912-f962bbfe7f72 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:42.164037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f303f8d-6c24-4152-9912-f962bbfe7f72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:42.164037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f303f8d-6c24-4152-9912-f962bbfe7f72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:42.166150 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f303f8d-6c24-4152-9912-f962bbfe7f72 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:42.188899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 210/423] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:42.200170 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f303f8d-6c24-4152-9912-f962bbfe7f72 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:42.286037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:42.310403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 214/424] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1049 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:42.318463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd86437-0d2a-46cc-b572-c5fd585d0439 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:42.420038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:42.576040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f303f8d-6c24-4152-9912-f962bbfe7f72 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:42.632036 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd86437-0d2a-46cc-b572-c5fd585d0439 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:42.717341 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da193e-dcb0-4333-bc3e-18a1a0fe45ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:42.718414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da193e-dcb0-4333-bc3e-18a1a0fe45ed None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:42.718860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da193e-dcb0-4333-bc3e-18a1a0fe45ed None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:42.719334 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da193e-dcb0-4333-bc3e-18a1a0fe45ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:42.747849 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5c5c8-a5e1-47c0-ba8d-b2e72d6a991c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:42.753211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5c5c8-a5e1-47c0-ba8d-b2e72d6a991c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:42.757983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5c5c8-a5e1-47c0-ba8d-b2e72d6a991c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:42.763589 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5c5c8-a5e1-47c0-ba8d-b2e72d6a991c None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:42.769103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5c5c8-a5e1-47c0-ba8d-b2e72d6a991c None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:42.772349 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5c5c8-a5e1-47c0-ba8d-b2e72d6a991c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:42.786085 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 215/425] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:42 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 468 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:42.789162 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca250b-bc81-4657-a203-b08b7e614db6 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:42.790048 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca250b-bc81-4657-a203-b08b7e614db6 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:42.790779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca250b-bc81-4657-a203-b08b7e614db6 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:42.794136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 216/426] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:42.798368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca250b-bc81-4657-a203-b08b7e614db6 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:42.910875 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da193e-dcb0-4333-bc3e-18a1a0fe45ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:42.919068 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da193e-dcb0-4333-bc3e-18a1a0fe45ed None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:42.963306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:43.040086 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca250b-bc81-4657-a203-b08b7e614db6 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:43.042858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:43.108254 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65bca0-243b-47cc-9ed7-09211dd68dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:43.109621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65bca0-243b-47cc-9ed7-09211dd68dcd None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:43.110246 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65bca0-243b-47cc-9ed7-09211dd68dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:43.110690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65bca0-243b-47cc-9ed7-09211dd68dcd None admin] PATH_INFO: `/v3/groups/86295264882f4e689435387a9f9025f0` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:43.112348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65bca0-243b-47cc-9ed7-09211dd68dcd None admin] RBAC: Authorizing `identity:get_group(group_id=86295264882f4e689435387a9f9025f0)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:43.112466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 211/427] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 913 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:43.113908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d29372-efdc-4a2a-8610-04b772aa697e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:43.114193 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d29372-efdc-4a2a-8610-04b772aa697e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:43.114463 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d29372-efdc-4a2a-8610-04b772aa697e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:43.114931 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65bca0-243b-47cc-9ed7-09211dd68dcd None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:43.122312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 217/428] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Aug 30 14:30:42 2022] GET /identity/v3/groups/86295264882f4e689435387a9f9025f0 => generated 226 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:43.124136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ba03f-172d-4060-b9f6-96c88a65fd95 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:43.125532 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ba03f-172d-4060-b9f6-96c88a65fd95 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:43.125842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ba03f-172d-4060-b9f6-96c88a65fd95 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:43.127431 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 218/429] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:43.128130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ba03f-172d-4060-b9f6-96c88a65fd95 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:43.136736 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f5d29372-efdc-4a2a-8610-04b772aa697e None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:43.138728 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f5d29372-efdc-4a2a-8610-04b772aa697e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:43.231867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 212/430] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:43.233468 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d29372-efdc-4a2a-8610-04b772aa697e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:43.273903 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ba03f-172d-4060-b9f6-96c88a65fd95 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:43.331766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:43.339587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02d359-4496-4c8c-ae97-a128f8293f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:43.340080 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:43.341084 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02d359-4496-4c8c-ae97-a128f8293f3c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:43.341549 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02d359-4496-4c8c-ae97-a128f8293f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:43.342049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02d359-4496-4c8c-ae97-a128f8293f3c None admin] PATH_INFO: `/v3/projects/e02f7e20d2404b3888cc2e57180a901b` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:43.343840 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02d359-4496-4c8c-ae97-a128f8293f3c None admin] RBAC: Authorizing `identity:get_project(project_id=e02f7e20d2404b3888cc2e57180a901b)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:43.345899 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02d359-4496-4c8c-ae97-a128f8293f3c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:43.353993 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 219/431] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:43 2022] GET /identity/v3/projects/e02f7e20d2404b3888cc2e57180a901b => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:43.354858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d29372-efdc-4a2a-8610-04b772aa697e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:43.355788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771738ed-e325-4bbb-b191-f74e2ac911bc None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:43.356238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771738ed-e325-4bbb-b191-f74e2ac911bc None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:43.356677 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771738ed-e325-4bbb-b191-f74e2ac911bc None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:43.358259 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 220/432] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:43.359247 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8124034-4f37-423b-bfbb-f563770d6ea0 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:43.359520 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8124034-4f37-423b-bfbb-f563770d6ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:43.359906 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8124034-4f37-423b-bfbb-f563770d6ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:43.383105 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-c8124034-4f37-423b-bfbb-f563770d6ea0 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:43.384422 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c8124034-4f37-423b-bfbb-f563770d6ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:43.412781 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf7e5b-ff70-4b30-9378-e08f6b61ac96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:43.413592 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf7e5b-ff70-4b30-9378-e08f6b61ac96 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:43.413894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf7e5b-ff70-4b30-9378-e08f6b61ac96 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:43.414135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf7e5b-ff70-4b30-9378-e08f6b61ac96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:43.414878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf7e5b-ff70-4b30-9378-e08f6b61ac96 None admin] RBAC: Adding query filter params (group.id=86295264882f4e689435387a9f9025f0, role.id=4eebd8098ab44bfc84cd3de29a679bdf, scope.project.id=e02f7e20d2404b3888cc2e57180a901b) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:43.415179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf7e5b-ff70-4b30-9378-e08f6b61ac96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:43.418051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf7e5b-ff70-4b30-9378-e08f6b61ac96 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:43.424144 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 213/433] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Aug 30 14:30:43 2022] GET /identity/v3/role_assignments?group.id=86295264882f4e689435387a9f9025f0&scope.project.id=e02f7e20d2404b3888cc2e57180a901b&role.id=4eebd8098ab44bfc84cd3de29a679bdf => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:43.425047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf7e5b-ff70-4b30-9378-e08f6b61ac96 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:43.553052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 221/434] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:43.558689 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d941da1-1dbd-4170-844b-4e3105ec264f None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:43.559718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d941da1-1dbd-4170-844b-4e3105ec264f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:43.560524 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d941da1-1dbd-4170-844b-4e3105ec264f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:43.623110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf7e5b-ff70-4b30-9378-e08f6b61ac96 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:43.623319 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-3d941da1-1dbd-4170-844b-4e3105ec264f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:43.623319 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-3d941da1-1dbd-4170-844b-4e3105ec264f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:43.693736 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141d243-67b2-4636-ab60-1a5c855665c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:43.694513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141d243-67b2-4636-ab60-1a5c855665c4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:43.694794 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141d243-67b2-4636-ab60-1a5c855665c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:43.696206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141d243-67b2-4636-ab60-1a5c855665c4 None admin] PATH_INFO: `/v3/groups/86295264882f4e689435387a9f9025f0` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:43.699316 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e141d243-67b2-4636-ab60-1a5c855665c4 None admin] RBAC: Authorizing `identity:get_group(group_id=86295264882f4e689435387a9f9025f0)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:43.701401 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e141d243-67b2-4636-ab60-1a5c855665c4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:43.709223 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 214/435] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Aug 30 14:30:43 2022] GET /identity/v3/groups/86295264882f4e689435387a9f9025f0 => generated 226 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:43.709223 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141d243-67b2-4636-ab60-1a5c855665c4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:43.774554 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:43.792222 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 222/436] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:43.796372 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d941da1-1dbd-4170-844b-4e3105ec264f None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:43.961245 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:43.964638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141d243-67b2-4636-ab60-1a5c855665c4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:44.019701 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d941da1-1dbd-4170-844b-4e3105ec264f None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:44.044585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ddc53d-65e0-4c31-9e2a-6f2de47e5f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:44.046132 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddc53d-65e0-4c31-9e2a-6f2de47e5f6f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:44.046929 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddc53d-65e0-4c31-9e2a-6f2de47e5f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:44.047833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddc53d-65e0-4c31-9e2a-6f2de47e5f6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:44.083243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ddc53d-65e0-4c31-9e2a-6f2de47e5f6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:44.096495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-a2ddc53d-65e0-4c31-9e2a-6f2de47e5f6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:44.096495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:44.096495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:44.096495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:44.096495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:44.096495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:44.096495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:44.096495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:44.096495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:44.096495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:44.096495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:44.096495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:44.096495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:44.096495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:44.096495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:44.096495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:44.098293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:44.098293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:44.098293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:44.098293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:44.098293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:44.098293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:44.098293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:44.098293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:44.098293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:44.098293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:44.098293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:44.098293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:44.098293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:44.098293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:44.098293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:44.098293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:44.099796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:44.099796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:44.099796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:44.099796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:44.099796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:44.099796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:44.099796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:44.099796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:44.099796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:44.099796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:44.099796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:30:44.099796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:44.101022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac628b4f-5630-4ae4-b053-47e4428f6d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:44.101022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac628b4f-5630-4ae4-b053-47e4428f6d1e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:44.101022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac628b4f-5630-4ae4-b053-47e4428f6d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:44.101022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac628b4f-5630-4ae4-b053-47e4428f6d1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:44.102342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac628b4f-5630-4ae4-b053-47e4428f6d1e None admin] RBAC: Adding query filter params (group.id=86295264882f4e689435387a9f9025f0, role.id=aedf4e1e5c7848099d7bd183778f5896, scope.project.id=e02f7e20d2404b3888cc2e57180a901b) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:44.102655 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac628b4f-5630-4ae4-b053-47e4428f6d1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:44.104438 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac628b4f-5630-4ae4-b053-47e4428f6d1e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:44.107495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 215/437] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:44.110771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ddc53d-65e0-4c31-9e2a-6f2de47e5f6f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:44.115894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 223/438] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Aug 30 14:30:43 2022] GET /identity/v3/role_assignments?group.id=86295264882f4e689435387a9f9025f0&scope.project.id=e02f7e20d2404b3888cc2e57180a901b&role.id=aedf4e1e5c7848099d7bd183778f5896 => generated 611 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:44.117326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac628b4f-5630-4ae4-b053-47e4428f6d1e None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:44.262119 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:44.284076 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac628b4f-5630-4ae4-b053-47e4428f6d1e None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:44.289292 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ddc53d-65e0-4c31-9e2a-6f2de47e5f6f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:44.328741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:44.347788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134d145-68fd-4090-ad1d-535253a88d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:44.348504 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134d145-68fd-4090-ad1d-535253a88d5f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:44.348933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134d145-68fd-4090-ad1d-535253a88d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:44.349351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134d145-68fd-4090-ad1d-535253a88d5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:44.353238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb580f-6519-4fa5-98a5-c4e4673e4f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:44.354540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb580f-6519-4fa5-98a5-c4e4673e4f3b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:44.355247 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb580f-6519-4fa5-98a5-c4e4673e4f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:44.355940 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb580f-6519-4fa5-98a5-c4e4673e4f3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:44.368645 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134d145-68fd-4090-ad1d-535253a88d5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:44.375132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-1134d145-68fd-4090-ad1d-535253a88d5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:44.375132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:44.375132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:44.375132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:44.375132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:44.375132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:44.375132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:44.375132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:44.375132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:44.375132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:44.375132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:44.375132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:44.375132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:44.375132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:44.375132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:44.375132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:44.377187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:44.377187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:44.377187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:44.377187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:44.377187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:44.377187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:44.377187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:44.377187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:44.377187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:44.377187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:44.377187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:44.377187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:44.377187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:44.377187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:44.377187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:44.377187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:44.378484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:44.378484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:44.378484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:44.378484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:44.378484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:44.378484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:44.378484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:44.378484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:44.378484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:44.378484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:44.378484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:44.378484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:44.379652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 224/439] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:44.379652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134d145-68fd-4090-ad1d-535253a88d5f None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:44.386563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb580f-6519-4fa5-98a5-c4e4673e4f3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:44.393557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-becb580f-6519-4fa5-98a5-c4e4673e4f3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:44.393557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:44.393557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:44.393557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:44.393557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:44.393557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:44.393557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:44.393557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:44.393557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:44.393557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:44.393557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:44.393557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:44.393557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:44.393557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:44.393557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:44.393557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:44.395302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:44.395302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:44.395302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:44.395302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:44.395302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:44.395302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:44.395302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:44.395302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:44.395302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:44.395302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:44.395302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:44.395302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:44.395302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:44.395302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:44.395302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:44.395302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:44.396874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:44.396874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:44.396874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:44.396874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:44.396874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:44.396874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:44.396874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:44.396874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:44.396874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:44.396874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:44.396874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:44.396874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:44.396874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 216/440] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:44.396874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb580f-6519-4fa5-98a5-c4e4673e4f3b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:44.560520 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134d145-68fd-4090-ad1d-535253a88d5f None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:44.563985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb580f-6519-4fa5-98a5-c4e4673e4f3b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:44.607228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463fb94-ec80-4f8a-835d-97960279a425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:44.607438 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6c534-7495-423b-b035-d41c6cee6fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:44.607846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6c534-7495-423b-b035-d41c6cee6fcc None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:44.608028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463fb94-ec80-4f8a-835d-97960279a425 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:44.608163 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6c534-7495-423b-b035-d41c6cee6fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:44.608311 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463fb94-ec80-4f8a-835d-97960279a425 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:44.608438 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6c534-7495-423b-b035-d41c6cee6fcc None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:44.608564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463fb94-ec80-4f8a-835d-97960279a425 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:44.609241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6c534-7495-423b-b035-d41c6cee6fcc None admin] RBAC: Adding query filter params (name=member) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:44.609887 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463fb94-ec80-4f8a-835d-97960279a425 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:44.609985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6c534-7495-423b-b035-d41c6cee6fcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:44.610056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463fb94-ec80-4f8a-835d-97960279a425 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:44.610819 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6c534-7495-423b-b035-d41c6cee6fcc None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:44.611150 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463fb94-ec80-4f8a-835d-97960279a425 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:44.628282 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 217/441] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:44.628508 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 225/442] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:44.628641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463fb94-ec80-4f8a-835d-97960279a425 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:44.629671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6c534-7495-423b-b035-d41c6cee6fcc None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:44.633020 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:44.717159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:44.756441 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463fb94-ec80-4f8a-835d-97960279a425 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:44.809181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c347567-abf7-4f20-8eda-5a3eac75c26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:44.810122 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c347567-abf7-4f20-8eda-5a3eac75c26c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:44.810627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c347567-abf7-4f20-8eda-5a3eac75c26c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:44.810803 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6c534-7495-423b-b035-d41c6cee6fcc None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:44.810955 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c347567-abf7-4f20-8eda-5a3eac75c26c None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:44.811522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c347567-abf7-4f20-8eda-5a3eac75c26c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:44.811826 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c347567-abf7-4f20-8eda-5a3eac75c26c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:44.813479 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c347567-abf7-4f20-8eda-5a3eac75c26c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:44.823307 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 218/443] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:44.824129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c347567-abf7-4f20-8eda-5a3eac75c26c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:44.871079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bd69e-31a9-4081-b471-022270332217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:44.871722 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bd69e-31a9-4081-b471-022270332217 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:44.872032 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bd69e-31a9-4081-b471-022270332217 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:44.872307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bd69e-31a9-4081-b471-022270332217 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:44.910189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:44.959286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c347567-abf7-4f20-8eda-5a3eac75c26c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:44.983536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701bd69e-31a9-4081-b471-022270332217 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:44.985419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701bd69e-31a9-4081-b471-022270332217 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:45.010238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93e32e-70d0-4807-b2cd-15e94e396742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:45.011129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93e32e-70d0-4807-b2cd-15e94e396742 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:45.011588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93e32e-70d0-4807-b2cd-15e94e396742 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:45.012298 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93e32e-70d0-4807-b2cd-15e94e396742 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:45.046720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:45.115286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93e32e-70d0-4807-b2cd-15e94e396742 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:45.116852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93e32e-70d0-4807-b2cd-15e94e396742 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:45.139467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 226/444] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:45.139467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bd69e-31a9-4081-b471-022270332217 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:45.214880 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:45.261835 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 219/445] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:45.262817 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93e32e-70d0-4807-b2cd-15e94e396742 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:45.286211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bd69e-31a9-4081-b471-022270332217 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:45.336664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e432c8c-bc47-457a-87d5-33c521e30d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:45.337302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e432c8c-bc47-457a-87d5-33c521e30d89 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:45.337580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e432c8c-bc47-457a-87d5-33c521e30d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:45.337852 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e432c8c-bc47-457a-87d5-33c521e30d89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:45.358774 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:45.408802 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93e32e-70d0-4807-b2cd-15e94e396742 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:45.440143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e432c8c-bc47-457a-87d5-33c521e30d89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:45.442677 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e432c8c-bc47-457a-87d5-33c521e30d89 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:45.462408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a372607-9b78-4933-938f-91b4219c4148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:45.463827 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a372607-9b78-4933-938f-91b4219c4148 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:45.464340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a372607-9b78-4933-938f-91b4219c4148 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:45.464802 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a372607-9b78-4933-938f-91b4219c4148 None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:45.466767 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a372607-9b78-4933-938f-91b4219c4148 None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:45.469382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a372607-9b78-4933-938f-91b4219c4148 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:45.472624 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 220/446] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:45 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:45.473885 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a372607-9b78-4933-938f-91b4219c4148 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:45.522369 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:45.579705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 227/447] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:45.584972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e432c8c-bc47-457a-87d5-33c521e30d89 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:45.623602 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a372607-9b78-4933-938f-91b4219c4148 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:45.650712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:45.685967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75fcc0-7e19-43fc-a1a3-2f96ea303ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:45.686718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75fcc0-7e19-43fc-a1a3-2f96ea303ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:45.687155 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75fcc0-7e19-43fc-a1a3-2f96ea303ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:45.687577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75fcc0-7e19-43fc-a1a3-2f96ea303ab7 None admin] PATH_INFO: `/v3/projects/f9ccb338f4e0425cad7e44cb87838ce1` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:45.689443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75fcc0-7e19-43fc-a1a3-2f96ea303ab7 None admin] RBAC: Authorizing `identity:get_project(project_id=f9ccb338f4e0425cad7e44cb87838ce1)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:45.692165 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75fcc0-7e19-43fc-a1a3-2f96ea303ab7 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:45.694501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 221/448] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:45 2022] GET /identity/v3/projects/f9ccb338f4e0425cad7e44cb87838ce1 => generated 344 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:45.695773 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75fcc0-7e19-43fc-a1a3-2f96ea303ab7 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:45.743704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e432c8c-bc47-457a-87d5-33c521e30d89 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:45.819042 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd0cad-9cdd-46cd-bcf1-9188e6dbb1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:45.820307 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75fcc0-7e19-43fc-a1a3-2f96ea303ab7 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:45.820495 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd0cad-9cdd-46cd-bcf1-9188e6dbb1bf None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:45.821143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd0cad-9cdd-46cd-bcf1-9188e6dbb1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:45.821790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd0cad-9cdd-46cd-bcf1-9188e6dbb1bf None admin] PATH_INFO: `/v3/groups/86295264882f4e689435387a9f9025f0` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:45.822181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:45.824644 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd0cad-9cdd-46cd-bcf1-9188e6dbb1bf None admin] RBAC: Authorizing `identity:get_group(group_id=86295264882f4e689435387a9f9025f0)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:45.827638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd0cad-9cdd-46cd-bcf1-9188e6dbb1bf None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:45.835858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 228/449] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Aug 30 14:30:45 2022] GET /identity/v3/groups/86295264882f4e689435387a9f9025f0 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:45.835858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd0cad-9cdd-46cd-bcf1-9188e6dbb1bf None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:45.865309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934b159-f003-4de1-bc3f-0d0b852fd6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:45.865966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934b159-f003-4de1-bc3f-0d0b852fd6ec None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:45.866238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934b159-f003-4de1-bc3f-0d0b852fd6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:45.866810 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934b159-f003-4de1-bc3f-0d0b852fd6ec None admin] PATH_INFO: `/v3/projects/b994c620a91d441aa453e5be1b958864` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:45.868018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934b159-f003-4de1-bc3f-0d0b852fd6ec None admin] RBAC: Authorizing `identity:get_project(project_id=b994c620a91d441aa453e5be1b958864)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:45.869470 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934b159-f003-4de1-bc3f-0d0b852fd6ec None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:45.871133 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 222/450] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:45 2022] GET /identity/v3/projects/b994c620a91d441aa453e5be1b958864 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:45.872993 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934b159-f003-4de1-bc3f-0d0b852fd6ec None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:45.983131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd0cad-9cdd-46cd-bcf1-9188e6dbb1bf None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:46.001514 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:46.021099 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934b159-f003-4de1-bc3f-0d0b852fd6ec None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:46.031905 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10493ad-7bdb-43bc-96d1-f429ce334cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:46.032602 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10493ad-7bdb-43bc-96d1-f429ce334cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:46.032877 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10493ad-7bdb-43bc-96d1-f429ce334cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:46.033142 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10493ad-7bdb-43bc-96d1-f429ce334cd6 None admin] PATH_INFO: `/v3/groups/1869d43de3f74a41844e5016f3f6a6a5` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:46.034370 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10493ad-7bdb-43bc-96d1-f429ce334cd6 None admin] RBAC: Authorizing `identity:get_group(group_id=1869d43de3f74a41844e5016f3f6a6a5)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:46.035813 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10493ad-7bdb-43bc-96d1-f429ce334cd6 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:46.037898 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 229/451] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Aug 30 14:30:45 2022] GET /identity/v3/groups/1869d43de3f74a41844e5016f3f6a6a5 => generated 229 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:46.038541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10493ad-7bdb-43bc-96d1-f429ce334cd6 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:46.070079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1209ab3-a0d2-4136-bfd8-029e886b8875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:46.071263 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1209ab3-a0d2-4136-bfd8-029e886b8875 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:46.071754 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1209ab3-a0d2-4136-bfd8-029e886b8875 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:46.072380 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1209ab3-a0d2-4136-bfd8-029e886b8875 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:46.073740 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1209ab3-a0d2-4136-bfd8-029e886b8875 None admin] RBAC: Adding query filter params (group.id=86295264882f4e689435387a9f9025f0, role.id=4eebd8098ab44bfc84cd3de29a679bdf, scope.project.id=b994c620a91d441aa453e5be1b958864) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:46.074282 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1209ab3-a0d2-4136-bfd8-029e886b8875 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:46.077142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1209ab3-a0d2-4136-bfd8-029e886b8875 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:46.096139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 223/452] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Aug 30 14:30:45 2022] GET /identity/v3/role_assignments?group.id=86295264882f4e689435387a9f9025f0&scope.project.id=b994c620a91d441aa453e5be1b958864&role.id=4eebd8098ab44bfc84cd3de29a679bdf => generated 611 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:46.096942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1209ab3-a0d2-4136-bfd8-029e886b8875 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:46.153575 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:46.235776 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10493ad-7bdb-43bc-96d1-f429ce334cd6 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:46.279983 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1209ab3-a0d2-4136-bfd8-029e886b8875 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:46.285617 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab920ae0-6167-4153-b791-28193df7e06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:46.286297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab920ae0-6167-4153-b791-28193df7e06c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:46.286568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab920ae0-6167-4153-b791-28193df7e06c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:46.286836 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab920ae0-6167-4153-b791-28193df7e06c None admin] PATH_INFO: `/v3/groups/86295264882f4e689435387a9f9025f0` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:46.288242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab920ae0-6167-4153-b791-28193df7e06c None admin] RBAC: Authorizing `identity:get_group(group_id=86295264882f4e689435387a9f9025f0)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:46.289728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab920ae0-6167-4153-b791-28193df7e06c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:46.297903 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 230/453] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Aug 30 14:30:46 2022] GET /identity/v3/groups/86295264882f4e689435387a9f9025f0 => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:46.299005 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab920ae0-6167-4153-b791-28193df7e06c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:46.350550 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab571aa-ba77-4f64-8913-c49e69e8a97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:46.351596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab571aa-ba77-4f64-8913-c49e69e8a97b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:46.352172 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab571aa-ba77-4f64-8913-c49e69e8a97b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:46.352542 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab571aa-ba77-4f64-8913-c49e69e8a97b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:46.354773 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab571aa-ba77-4f64-8913-c49e69e8a97b None admin] RBAC: Adding query filter params (group.id=1869d43de3f74a41844e5016f3f6a6a5, role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=f9ccb338f4e0425cad7e44cb87838ce1) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:46.355279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab571aa-ba77-4f64-8913-c49e69e8a97b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:46.357911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab571aa-ba77-4f64-8913-c49e69e8a97b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:46.367446 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 224/454] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Aug 30 14:30:46 2022] GET /identity/v3/role_assignments?group.id=1869d43de3f74a41844e5016f3f6a6a5&scope.project.id=f9ccb338f4e0425cad7e44cb87838ce1&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 611 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:46.396472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:46.498824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab920ae0-6167-4153-b791-28193df7e06c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:46.555807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4154f-a9a9-40f6-82b4-cf91ab6e6833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:46.556476 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4154f-a9a9-40f6-82b4-cf91ab6e6833 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:46.557183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4154f-a9a9-40f6-82b4-cf91ab6e6833 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:46.557183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4154f-a9a9-40f6-82b4-cf91ab6e6833 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:46.557776 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4154f-a9a9-40f6-82b4-cf91ab6e6833 None admin] RBAC: Adding query filter params (group.id=86295264882f4e689435387a9f9025f0, role.id=aedf4e1e5c7848099d7bd183778f5896, scope.project.id=b994c620a91d441aa453e5be1b958864) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:46.558081 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4154f-a9a9-40f6-82b4-cf91ab6e6833 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:46.560172 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4154f-a9a9-40f6-82b4-cf91ab6e6833 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:46.570436 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 231/455] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Aug 30 14:30:46 2022] GET /identity/v3/role_assignments?group.id=86295264882f4e689435387a9f9025f0&scope.project.id=b994c620a91d441aa453e5be1b958864&role.id=aedf4e1e5c7848099d7bd183778f5896 => generated 611 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:49.719543 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671295b-5b97-4497-9355-76a3b1c42fca None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:49.720096 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671295b-5b97-4497-9355-76a3b1c42fca None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:49.720096 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671295b-5b97-4497-9355-76a3b1c42fca None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:49.722767 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 225/456] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:49.734957 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3c3fd-516b-4427-88f0-fb21b4233acd None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:49.735810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3c3fd-516b-4427-88f0-fb21b4233acd None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:49.735810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3c3fd-516b-4427-88f0-fb21b4233acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:49.760307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-06c3c3fd-516b-4427-88f0-fb21b4233acd None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:49.762380 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-06c3c3fd-516b-4427-88f0-fb21b4233acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:49.883062 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 232/457] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:50.391816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f6221-c55f-475c-ab7a-d40f9033eb25 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:50.392339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f6221-c55f-475c-ab7a-d40f9033eb25 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:50.392478 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f6221-c55f-475c-ab7a-d40f9033eb25 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:50.393853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 226/458] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:30:50.404040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e303629-f744-4e5d-8f11-f140f6ec4947 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:50.404040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e303629-f744-4e5d-8f11-f140f6ec4947 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:50.404040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e303629-f744-4e5d-8f11-f140f6ec4947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:50.430159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-9e303629-f744-4e5d-8f11-f140f6ec4947 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:50.431510 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-9e303629-f744-4e5d-8f11-f140f6ec4947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:50.487888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:50.589171 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 233/459] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:50.622892 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f6221-c55f-475c-ab7a-d40f9033eb25 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:50.940908 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82289d01-28ec-4c03-9677-44a5744632c2 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:50.942426 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82289d01-28ec-4c03-9677-44a5744632c2 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:50.943180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82289d01-28ec-4c03-9677-44a5744632c2 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:50.948038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:50.951475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 234/460] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:50 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:50.959110 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbb047-3e1a-4ba7-acf8-c5eafb3c9c30 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:50.959964 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbb047-3e1a-4ba7-acf8-c5eafb3c9c30 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:50.968097 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbb047-3e1a-4ba7-acf8-c5eafb3c9c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:50.973155 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f6221-c55f-475c-ab7a-d40f9033eb25 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:51.015278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-34dbb047-3e1a-4ba7-acf8-c5eafb3c9c30 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:51.017679 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-34dbb047-3e1a-4ba7-acf8-c5eafb3c9c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:51.054521 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c2e3b-6137-41ad-912c-8b462badd52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:51.056126 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c2e3b-6137-41ad-912c-8b462badd52e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:51.058329 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c2e3b-6137-41ad-912c-8b462badd52e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:51.059126 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c2e3b-6137-41ad-912c-8b462badd52e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:51.100543 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c2e3b-6137-41ad-912c-8b462badd52e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:51.103699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c2e3b-6137-41ad-912c-8b462badd52e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:51.145638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-652c2e3b-6137-41ad-912c-8b462badd52e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:51.145638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:51.145638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:51.145638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:51.145638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:51.145638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:51.145638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:51.145638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:51.145638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:51.145638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:51.145638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:51.145638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:51.145638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:51.145638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:51.145638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:51.145638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:51.147348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:51.147348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:51.147348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:51.147348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:51.147348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:51.147348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:51.147348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:51.147348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:51.147348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:51.147348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:51.147348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:51.147348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:51.147348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:51.147348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:51.147348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:51.147348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:51.147348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:51.148664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:51.148664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:51.148664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:51.152684 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 227/461] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 530 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:51.153141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c2e3b-6137-41ad-912c-8b462badd52e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:51.213302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 235/462] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:51.215130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbb047-3e1a-4ba7-acf8-c5eafb3c9c30 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:51.363254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c2e3b-6137-41ad-912c-8b462badd52e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:51.439059 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea16e13-ef83-4dc4-92f6-1607e620f00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:51.440132 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea16e13-ef83-4dc4-92f6-1607e620f00a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:51.440606 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea16e13-ef83-4dc4-92f6-1607e620f00a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:51.441063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea16e13-ef83-4dc4-92f6-1607e620f00a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:51.484536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea16e13-ef83-4dc4-92f6-1607e620f00a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:51.487613 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea16e13-ef83-4dc4-92f6-1607e620f00a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:51.542856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-cea16e13-ef83-4dc4-92f6-1607e620f00a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:51.542856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:51.542856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:51.542856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:51.542856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:51.542856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:51.542856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:51.542856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:51.542856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:51.542856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:51.542856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:51.542856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:51.542856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:51.542856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:51.542856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:51.542856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:51.544636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:51.544636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:51.544636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:51.544636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:51.544636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:51.544636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:51.544636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:51.544636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:51.544636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:51.544636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:51.544636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:51.544636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:51.544636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:51.544636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:51.544636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:51.544636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:51.544636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:51.546283 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:51.546283 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:51.546283 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:51.550108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 228/463] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:51.552157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea16e13-ef83-4dc4-92f6-1607e620f00a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:51.560177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbb047-3e1a-4ba7-acf8-c5eafb3c9c30 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:51.562253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:51.564735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:51.647167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb484b-a751-4f43-a708-c7e0e68136ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:51.649039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb484b-a751-4f43-a708-c7e0e68136ef None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:51.649448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb484b-a751-4f43-a708-c7e0e68136ef None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:51.649858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb484b-a751-4f43-a708-c7e0e68136ef None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:51.651020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb484b-a751-4f43-a708-c7e0e68136ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:51.651583 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb484b-a751-4f43-a708-c7e0e68136ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:51.654226 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb484b-a751-4f43-a708-c7e0e68136ef None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:51.679218 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 236/464] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 465 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:51.680902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb484b-a751-4f43-a708-c7e0e68136ef None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:51.802107 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea16e13-ef83-4dc4-92f6-1607e620f00a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:51.869756 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edbf5e-d049-4ea2-8a36-7238b63eb5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:51.871253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edbf5e-d049-4ea2-8a36-7238b63eb5af None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:51.872061 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edbf5e-d049-4ea2-8a36-7238b63eb5af None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:51.872933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edbf5e-d049-4ea2-8a36-7238b63eb5af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:51.894574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edbf5e-d049-4ea2-8a36-7238b63eb5af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:51.897677 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edbf5e-d049-4ea2-8a36-7238b63eb5af None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:51.899807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb484b-a751-4f43-a708-c7e0e68136ef None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:51.928547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-29edbf5e-d049-4ea2-8a36-7238b63eb5af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:51.928547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:51.928547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:51.928547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:51.928547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:51.928547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:51.928547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:51.928547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:51.928547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:51.928547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:51.928547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:51.928547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:51.928547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:51.928547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:51.928547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:51.928547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:51.930488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:51.930488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:51.930488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:51.930488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:51.930488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:51.930488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:51.930488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:51.930488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:51.930488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:51.930488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:51.930488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:51.930488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:51.930488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:51.930488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:51.930488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:51.930488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:51.930488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:51.932158 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:51.932158 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:51.932158 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:51.935347 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 229/465] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:51.937210 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edbf5e-d049-4ea2-8a36-7238b63eb5af None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:51.983681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6a4c0-a7ec-439a-945a-290dab97a158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:51.984807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6a4c0-a7ec-439a-945a-290dab97a158 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:51.985278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6a4c0-a7ec-439a-945a-290dab97a158 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:51.985757 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6a4c0-a7ec-439a-945a-290dab97a158 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:51.986933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6a4c0-a7ec-439a-945a-290dab97a158 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:51.987468 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6a4c0-a7ec-439a-945a-290dab97a158 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:51.991021 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6a4c0-a7ec-439a-945a-290dab97a158 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:52.023949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 237/466] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:52.026243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6a4c0-a7ec-439a-945a-290dab97a158 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:52.056742 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:52.089059 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:52.165861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edbf5e-d049-4ea2-8a36-7238b63eb5af None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:52.222273 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fed8c-cca4-47a4-b0a7-481b7111db94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:52.222942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fed8c-cca4-47a4-b0a7-481b7111db94 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:52.223213 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fed8c-cca4-47a4-b0a7-481b7111db94 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:52.223479 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fed8c-cca4-47a4-b0a7-481b7111db94 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:52.224205 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6fed8c-cca4-47a4-b0a7-481b7111db94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:52.225953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6fed8c-cca4-47a4-b0a7-481b7111db94 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:52.252977 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6a4c0-a7ec-439a-945a-290dab97a158 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:52.257277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 230/467] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:30:51 2022] POST /identity/v3/users => generated 246 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:52.258780 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fed8c-cca4-47a4-b0a7-481b7111db94 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:52.348384 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f83e07-2f93-4aef-9f2e-db238ee7403b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:52.349415 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f83e07-2f93-4aef-9f2e-db238ee7403b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:52.349858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f83e07-2f93-4aef-9f2e-db238ee7403b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:52.350344 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f83e07-2f93-4aef-9f2e-db238ee7403b None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:52.351614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f83e07-2f93-4aef-9f2e-db238ee7403b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:52.352194 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f83e07-2f93-4aef-9f2e-db238ee7403b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:52.354977 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f83e07-2f93-4aef-9f2e-db238ee7403b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:52.385203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 238/468] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:52.389282 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a16db-e110-475a-a3d3-715ea10fcf33 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:52.390852 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a16db-e110-475a-a3d3-715ea10fcf33 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:52.391703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a16db-e110-475a-a3d3-715ea10fcf33 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:52.396637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 239/469] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:52.399046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a16db-e110-475a-a3d3-715ea10fcf33 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:52.488226 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fed8c-cca4-47a4-b0a7-481b7111db94 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:52.506535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:52.536712 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d5c5c-e8c5-4011-b09f-b23b4b081f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:52.537352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d5c5c-e8c5-4011-b09f-b23b4b081f1b None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:52.537636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d5c5c-e8c5-4011-b09f-b23b4b081f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:52.537910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d5c5c-e8c5-4011-b09f-b23b4b081f1b None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:52.538697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d5c5c-e8c5-4011-b09f-b23b4b081f1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:52.540355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d5c5c-e8c5-4011-b09f-b23b4b081f1b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:52.572111 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 231/470] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:30:52 2022] POST /identity/v3/users => generated 248 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:52.572396 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f0ee2-2c6f-4673-b8a4-4c49d465a380 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:52.573106 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f0ee2-2c6f-4673-b8a4-4c49d465a380 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:52.573106 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f0ee2-2c6f-4673-b8a4-4c49d465a380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:52.592836 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:52.597672 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-488f0ee2-2c6f-4673-b8a4-4c49d465a380 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:52.599704 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-488f0ee2-2c6f-4673-b8a4-4c49d465a380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:52.622683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a16db-e110-475a-a3d3-715ea10fcf33 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:52.684398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099c486-84b8-4a5c-84db-4e78fa5e9ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:52.685356 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099c486-84b8-4a5c-84db-4e78fa5e9ced None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:52.685795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099c486-84b8-4a5c-84db-4e78fa5e9ced None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:52.686220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099c486-84b8-4a5c-84db-4e78fa5e9ced None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:52.687406 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099c486-84b8-4a5c-84db-4e78fa5e9ced None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:52.689139 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099c486-84b8-4a5c-84db-4e78fa5e9ced None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:52.729974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 240/471] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:30:52 2022] POST /identity/v3/users => generated 248 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:52.730826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607dd40-6915-4078-9389-00bd45deb9aa None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:52.734277 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607dd40-6915-4078-9389-00bd45deb9aa None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:52.734795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607dd40-6915-4078-9389-00bd45deb9aa None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:52.739402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 241/472] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:52.747746 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d749d-4f80-4b25-95a8-352a58d6fe02 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:52.748222 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d749d-4f80-4b25-95a8-352a58d6fe02 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:52.748693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d749d-4f80-4b25-95a8-352a58d6fe02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:52.775387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 232/473] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:52.810039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-0b3d749d-4f80-4b25-95a8-352a58d6fe02 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:52.812517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0b3d749d-4f80-4b25-95a8-352a58d6fe02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:52.940098 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 242/474] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:30:53.238119 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f0ee2-2c6f-4673-b8a4-4c49d465a380 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:53.241809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d749d-4f80-4b25-95a8-352a58d6fe02 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:53.389428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:53.432787 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d749d-4f80-4b25-95a8-352a58d6fe02 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:53.474811 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9ea99-028c-45bc-b3a4-c43e95e0b2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:53.475478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9ea99-028c-45bc-b3a4-c43e95e0b2ea None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:53.475760 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9ea99-028c-45bc-b3a4-c43e95e0b2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:53.476052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9ea99-028c-45bc-b3a4-c43e95e0b2ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:53.488977 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9ea99-028c-45bc-b3a4-c43e95e0b2ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:53.490143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9ea99-028c-45bc-b3a4-c43e95e0b2ea None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:53.508492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-3ec9ea99-028c-45bc-b3a4-c43e95e0b2ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:53.508492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:53.508492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:53.508492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:53.508492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:53.508492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:53.508492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:53.508492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:53.508492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:53.508492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:53.508492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:53.508492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:53.508492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:53.508492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:53.508492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:53.508492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:53.510413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:53.510413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:53.510413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:53.510413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:53.510413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:53.510413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:53.510413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:53.510413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:53.510413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:53.510413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:53.510413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:53.510413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:53.510413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:53.510413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:53.510413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:53.510413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:53.510413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:53.512295 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:53.512295 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:53.512295 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:53.520115 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 243/475] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:53.527866 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9ea99-028c-45bc-b3a4-c43e95e0b2ea None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:53.572281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:53.593919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f0ee2-2c6f-4673-b8a4-4c49d465a380 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:53.724427 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9ea99-028c-45bc-b3a4-c43e95e0b2ea None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:53.753762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a205fa2-f53d-4e89-977f-1afa6ac52322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:53.754676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a205fa2-f53d-4e89-977f-1afa6ac52322 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:53.755182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a205fa2-f53d-4e89-977f-1afa6ac52322 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:53.755658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a205fa2-f53d-4e89-977f-1afa6ac52322 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:53.756963 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a205fa2-f53d-4e89-977f-1afa6ac52322 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:53.759446 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a205fa2-f53d-4e89-977f-1afa6ac52322 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:53.772788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:53.779322 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba6f5c-e277-4d69-bb69-98b8e4c082e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:53.780075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba6f5c-e277-4d69-bb69-98b8e4c082e6 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:53.780359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba6f5c-e277-4d69-bb69-98b8e4c082e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:53.780791 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba6f5c-e277-4d69-bb69-98b8e4c082e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:53.782160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba6f5c-e277-4d69-bb69-98b8e4c082e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:53.782592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba6f5c-e277-4d69-bb69-98b8e4c082e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:53.784171 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.sql.core [None req-9a205fa2-f53d-4e89-977f-1afa6ac52322 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 14:30:53.784171 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 14:30:53.784171 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [parameters: {'id': 'bc0c2ba34eb84720a61d69a5563bb6cc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 14:30:53.784171 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81732) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 14:30:53.794755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba6f5c-e277-4d69-bb69-98b8e4c082e6 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:53.817135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-9a205fa2-f53d-4e89-977f-1afa6ac52322 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 14:30:53.817135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:53.817135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:30:53.817135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:30:53.817135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:30:53.817135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:30:53.817135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:30:53.817135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:30:53.817135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:30:53.817135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:30:53.817135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:30:53.817135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:30:53.817135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:30:53.817135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application result.read() Aug 30 14:30:53.817135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return ref.to_dict() Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:30:53.818900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:30:53.820765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 14:30:53.820765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application yield resource Aug 30 14:30:53.820765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:30:53.820765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:30:53.820765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 14:30:53.820765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 14:30:53.820765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 14:30:53.820765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 14:30:53.820765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 14:30:53.820765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise self.value Aug 30 14:30:53.820765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 14:30:53.820765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 14:30:53.820765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 14:30:53.820765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application session.commit() Aug 30 14:30:53.820765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 14:30:53.820765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 14:30:53.820765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 14:30:53.823964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 14:30:53.823964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 14:30:53.823964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self.session.flush() Aug 30 14:30:53.823964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 14:30:53.823964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 14:30:53.823964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 14:30:53.823964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 14:30:53.823964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 14:30:53.823964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application compat.raise_( Aug 30 14:30:53.823964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:30:53.823964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception Aug 30 14:30:53.823964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 14:30:53.823964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 14:30:53.823964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 14:30:53.823964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 14:30:53.823964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 14:30:53.823964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 14:30:53.833356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 14:30:53.833356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 14:30:53.833356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 14:30:53.833356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 14:30:53.833356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 14:30:53.833356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 14:30:53.833356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 14:30:53.833356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 14:30:53.833356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 14:30:53.833356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 14:30:53.833356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 14:30:53.833356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 14:30:53.833356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 14:30:53.833356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 14:30:53.833356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:30:53.833356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception Aug 30 14:30:53.836535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:30:53.836535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:30:53.836535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:30:53.836535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:30:53.836535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:30:53.836535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:30:53.836535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:30:53.836535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:30:53.836535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:30:53.836535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:30:53.836535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:30:53.836535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application result.read() Aug 30 14:30:53.836535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:30:53.836535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:30:53.836535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:30:53.836535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:30:53.846324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:30:53.846324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:30:53.846324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:30:53.846324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:30:53.846324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 14:30:53.846324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 14:30:53.846324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [parameters: {'id': 'bc0c2ba34eb84720a61d69a5563bb6cc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 14:30:53.846324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 14:30:53.846324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:53.846324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:53.846324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:53.846324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:53.846324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:53.846324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:53.846324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:53.846324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:53.849196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:53.849196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:53.849196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:53.849196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:53.849196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:53.849196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:53.849196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 30 14:30:53.849196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 30 14:30:53.849196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:53.849196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:53.849196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 30 14:30:53.849196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 30 14:30:53.849196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 14:30:53.849196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 14:30:53.849196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 14:30:53.849196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:53.856224 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 244/477] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:53.856529 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 233/477] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Aug 30 14:30:53 2022] POST /identity/v3/roles => generated 144 bytes in 589 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:53.856763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba6f5c-e277-4d69-bb69-98b8e4c082e6 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:53.856996 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a205fa2-f53d-4e89-977f-1afa6ac52322 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:53.990631 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba6f5c-e277-4d69-bb69-98b8e4c082e6 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:54.032781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0d205-1fd7-43c7-9342-407f8fb44dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:54.033529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0d205-1fd7-43c7-9342-407f8fb44dbb None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:54.033837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0d205-1fd7-43c7-9342-407f8fb44dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:54.034103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0d205-1fd7-43c7-9342-407f8fb44dbb None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:54.034795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0d205-1fd7-43c7-9342-407f8fb44dbb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:54.036317 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0d205-1fd7-43c7-9342-407f8fb44dbb None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:54.068596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 245/478] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:30:53 2022] POST /identity/v3/users => generated 249 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:30:54.126723 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a205fa2-f53d-4e89-977f-1afa6ac52322 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:54.183798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86438e49-89e0-4be0-8df0-a7533f00e121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:54.184488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86438e49-89e0-4be0-8df0-a7533f00e121 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:54.184769 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86438e49-89e0-4be0-8df0-a7533f00e121 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:54.185036 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86438e49-89e0-4be0-8df0-a7533f00e121 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:54.206493 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86438e49-89e0-4be0-8df0-a7533f00e121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:54.211759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-86438e49-89e0-4be0-8df0-a7533f00e121 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:54.211759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:54.211759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:54.211759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:54.211759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:54.211759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:54.211759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:54.211759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:54.211759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:54.211759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:54.211759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:54.211759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:54.211759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:54.211759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:54.211759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:54.211759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:54.215025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:54.215025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:54.215025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:54.215025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:54.215025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:54.215025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:54.215025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:54.215025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:54.215025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:54.215025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:54.215025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:54.215025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:54.215025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:54.215025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:54.215025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:54.215025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:54.218123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:54.218123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:54.218123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:54.218123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:54.218123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:54.218123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:54.218123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:54.218123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:54.218123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:54.218123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:54.218123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:30:54.218123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:54.218123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 234/479] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:54.229160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0d205-1fd7-43c7-9342-407f8fb44dbb None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:54.288689 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:54.436026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0d205-1fd7-43c7-9342-407f8fb44dbb None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:54.512603 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d230f9-8206-4085-9289-5aa60711a2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:54.514169 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d230f9-8206-4085-9289-5aa60711a2dc None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:54.514976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d230f9-8206-4085-9289-5aa60711a2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:54.515732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d230f9-8206-4085-9289-5aa60711a2dc None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:54.517255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d230f9-8206-4085-9289-5aa60711a2dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:54.518078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d230f9-8206-4085-9289-5aa60711a2dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:54.520971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d230f9-8206-4085-9289-5aa60711a2dc None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:54.539265 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 246/480] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:54.899480 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f3a3c-4e6c-4a11-b200-458cb8f1658a None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:54.900033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f3a3c-4e6c-4a11-b200-458cb8f1658a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:54.900033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f3a3c-4e6c-4a11-b200-458cb8f1658a None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:54.901736 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 235/481] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:54.907796 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980d4c4-9796-4696-b38e-a9becc7c6be5 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:54.908155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980d4c4-9796-4696-b38e-a9becc7c6be5 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:54.908374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980d4c4-9796-4696-b38e-a9becc7c6be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:54.931948 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-6980d4c4-9796-4696-b38e-a9becc7c6be5 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:54.933552 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6980d4c4-9796-4696-b38e-a9becc7c6be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:55.020787 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 247/482] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:30:55.537732 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f3a3c-4e6c-4a11-b200-458cb8f1658a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:55.576693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:55.723241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f3a3c-4e6c-4a11-b200-458cb8f1658a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:55.788752 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049f05f-c665-400d-b5a5-e98b8bb90ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:55.789725 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049f05f-c665-400d-b5a5-e98b8bb90ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:55.790197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049f05f-c665-400d-b5a5-e98b8bb90ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:55.790658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049f05f-c665-400d-b5a5-e98b8bb90ca1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:55.819197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049f05f-c665-400d-b5a5-e98b8bb90ca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:55.828904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-c049f05f-c665-400d-b5a5-e98b8bb90ca1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:30:55.828904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:55.828904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:55.828904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:55.828904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:55.828904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:55.828904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:55.828904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:55.828904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:55.828904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:55.828904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:55.828904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:55.828904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:55.828904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:55.828904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:55.828904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:55.830643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:55.830643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:55.830643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:55.830643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:55.830643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:55.830643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:55.830643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:55.830643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:55.830643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:55.830643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:55.830643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:55.830643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:55.830643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:55.830643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:55.830643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:55.830643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:55.831786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:55.831786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:55.831786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:55.831786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:55.831786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:55.831786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:55.831786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:55.831786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:55.831786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:55.831786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:55.831786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:30:55.831786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:55.831786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 236/483] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:30:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:55.836632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499946e-7c26-4a2a-b84e-fe66e084cba5 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:55.837934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499946e-7c26-4a2a-b84e-fe66e084cba5 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:55.838094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049f05f-c665-400d-b5a5-e98b8bb90ca1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:55.838969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499946e-7c26-4a2a-b84e-fe66e084cba5 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:55.849707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 248/484] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:55 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:55.861224 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc688e-344e-4cf2-a62a-38cc25eb23c9 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:55.862106 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc688e-344e-4cf2-a62a-38cc25eb23c9 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:55.862849 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc688e-344e-4cf2-a62a-38cc25eb23c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:55.915990 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a1fc688e-344e-4cf2-a62a-38cc25eb23c9 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:55.918437 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a1fc688e-344e-4cf2-a62a-38cc25eb23c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:55.976335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:55.978648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:56.003540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049f05f-c665-400d-b5a5-e98b8bb90ca1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:56.045449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a4266-38dc-4132-994d-829eac1ef888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:56.045973 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a4266-38dc-4132-994d-829eac1ef888 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:56.046303 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a4266-38dc-4132-994d-829eac1ef888 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:56.046574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a4266-38dc-4132-994d-829eac1ef888 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:56.047276 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a4266-38dc-4132-994d-829eac1ef888 None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:56.047577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a4266-38dc-4132-994d-829eac1ef888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:56.049044 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a4266-38dc-4132-994d-829eac1ef888 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:56.059057 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 237/485] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:30:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:56.060495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d8deb-7a40-4481-8387-0a175af4f151 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:56.060774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d8deb-7a40-4481-8387-0a175af4f151 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:56.061042 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d8deb-7a40-4481-8387-0a175af4f151 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:56.063202 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 238/486] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:56.071277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2573002-30bc-45fc-b2e1-e2fb91a52ce9 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:56.071762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2573002-30bc-45fc-b2e1-e2fb91a52ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:56.072310 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2573002-30bc-45fc-b2e1-e2fb91a52ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:56.094087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 249/487] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:56.095957 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc688e-344e-4cf2-a62a-38cc25eb23c9 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:56.121923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e2573002-30bc-45fc-b2e1-e2fb91a52ce9 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:56.124226 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e2573002-30bc-45fc-b2e1-e2fb91a52ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:56.362923 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc688e-344e-4cf2-a62a-38cc25eb23c9 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:56.366560 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 239/488] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:56.425649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d88ce-95aa-4975-9dd7-007f8363b51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:56.426750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d88ce-95aa-4975-9dd7-007f8363b51e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:56.427232 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d88ce-95aa-4975-9dd7-007f8363b51e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:56.427704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d88ce-95aa-4975-9dd7-007f8363b51e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:56.448439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d88ce-95aa-4975-9dd7-007f8363b51e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:56.453236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d88ce-95aa-4975-9dd7-007f8363b51e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:56.470756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-486d88ce-95aa-4975-9dd7-007f8363b51e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:56.470756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:56.470756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:56.470756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:56.470756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:56.470756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:56.470756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:56.470756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:56.470756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:56.470756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:56.470756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:56.470756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:56.470756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:56.470756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:56.470756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:56.470756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:56.472525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:56.472525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:56.472525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:56.472525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:56.472525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:56.472525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:56.472525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:56.472525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:56.472525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:56.472525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:56.472525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:56.472525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:56.472525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:56.472525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:56.472525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:56.472525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:56.472525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:56.474118 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:56.474118 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:56.474118 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:56.474118 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 250/489] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:56.478952 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2573002-30bc-45fc-b2e1-e2fb91a52ce9 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:56.592437 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:56.625448 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2573002-30bc-45fc-b2e1-e2fb91a52ce9 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:56.666229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e179f-86ef-454e-99ba-6576259e5e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:56.667247 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e179f-86ef-454e-99ba-6576259e5e9b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:56.667762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e179f-86ef-454e-99ba-6576259e5e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:56.668235 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e179f-86ef-454e-99ba-6576259e5e9b None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:56.669133 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e179f-86ef-454e-99ba-6576259e5e9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:56.669666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e179f-86ef-454e-99ba-6576259e5e9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:56.670925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d88ce-95aa-4975-9dd7-007f8363b51e None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:56.671825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e179f-86ef-454e-99ba-6576259e5e9b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:56.685945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:56.692211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 240/490] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:56.697858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e179f-86ef-454e-99ba-6576259e5e9b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:56.817614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d88ce-95aa-4975-9dd7-007f8363b51e None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:56.817847 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e179f-86ef-454e-99ba-6576259e5e9b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:56.856316 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:56.870179 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc145c8-c6db-4f4f-9a56-a02af6dfe672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:56.872187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc145c8-c6db-4f4f-9a56-a02af6dfe672 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:56.872670 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc145c8-c6db-4f4f-9a56-a02af6dfe672 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:56.873134 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc145c8-c6db-4f4f-9a56-a02af6dfe672 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:56.877564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efecbb9-d267-458b-8ccb-a447c1e761ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:56.878037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efecbb9-d267-458b-8ccb-a447c1e761ed None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:56.878319 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efecbb9-d267-458b-8ccb-a447c1e761ed None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:56.878594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efecbb9-d267-458b-8ccb-a447c1e761ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:56.891486 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc145c8-c6db-4f4f-9a56-a02af6dfe672 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:56.895425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efecbb9-d267-458b-8ccb-a447c1e761ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:56.897237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efecbb9-d267-458b-8ccb-a447c1e761ed None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:56.898696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-5dc145c8-c6db-4f4f-9a56-a02af6dfe672 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:30:56.898696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:56.898696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:56.898696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:56.898696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:56.898696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:56.898696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:56.898696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:56.898696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:56.898696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:56.898696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:56.898696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:56.898696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:56.898696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:56.898696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:56.898696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:56.899605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:56.899605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:56.899605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:56.899605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:56.899605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:56.899605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:56.899605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:56.899605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:56.899605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:56.899605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:56.899605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:56.899605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:56.899605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:56.899605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:56.899605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:56.899605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:56.900469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:56.900469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:56.900469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:56.900469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:56.900469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:56.900469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:56.900469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:56.900469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:56.900469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:56.900469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:56.900469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:30:56.900469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:56.901150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 251/491] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:30:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:56.901721 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc145c8-c6db-4f4f-9a56-a02af6dfe672 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:56.923076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-6efecbb9-d267-458b-8ccb-a447c1e761ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:30:56.923076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:56.923076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:56.923076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:56.923076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:56.923076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:56.923076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:56.923076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:56.923076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:56.923076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:56.923076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:56.923076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:56.923076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:56.923076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:56.923076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:56.923076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:56.924116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:56.925018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:56.925018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:56.925018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:56.925018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:56.925018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:56.925018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:56.925018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:56.925018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:56.925018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:30:56.925018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:56.925018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 241/492] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Aug 30 14:30:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:56.925588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efecbb9-d267-458b-8ccb-a447c1e761ed None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:57.014034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:57.052866 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc145c8-c6db-4f4f-9a56-a02af6dfe672 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:57.067578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efecbb9-d267-458b-8ccb-a447c1e761ed None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:57.094406 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323556b4-bdbf-4ccb-9f2e-b5c9f105062d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:57.095571 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323556b4-bdbf-4ccb-9f2e-b5c9f105062d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.096102 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323556b4-bdbf-4ccb-9f2e-b5c9f105062d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.096614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323556b4-bdbf-4ccb-9f2e-b5c9f105062d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.108643 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323556b4-bdbf-4ccb-9f2e-b5c9f105062d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:57.113800 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee155f6-e879-4b9a-b0f3-9daccae62d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:57.114481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee155f6-e879-4b9a-b0f3-9daccae62d5e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.114801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee155f6-e879-4b9a-b0f3-9daccae62d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.115073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee155f6-e879-4b9a-b0f3-9daccae62d5e None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.115759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee155f6-e879-4b9a-b0f3-9daccae62d5e None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:57.116090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee155f6-e879-4b9a-b0f3-9daccae62d5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:57.116455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-323556b4-bdbf-4ccb-9f2e-b5c9f105062d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:30:57.116455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:57.116455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:57.116455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:57.116455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:57.116455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:57.116455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:57.116455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:57.116455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:57.116455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:57.116455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:57.116455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:57.116455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:57.116455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:57.116455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:57.116455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:57.117313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:57.117313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:57.117313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:57.117313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:57.117313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:57.117313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:57.117313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:57.117313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:57.117313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:57.117313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:57.117313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:57.117313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:57.117313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:57.117313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:57.117313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:57.117313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:57.118128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:57.118128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:57.118128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:57.118128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:57.118128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:57.118128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:57.118128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:57.118128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:57.118128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:57.118128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:57.118128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:30:57.118128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:57.118738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee155f6-e879-4b9a-b0f3-9daccae62d5e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:57.123021 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 252/493] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:30:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:57.127449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323556b4-bdbf-4ccb-9f2e-b5c9f105062d None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:57.128064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 242/494] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:30:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:57.128965 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee155f6-e879-4b9a-b0f3-9daccae62d5e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:57.205565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:57.266892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323556b4-bdbf-4ccb-9f2e-b5c9f105062d None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:57.276754 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee155f6-e879-4b9a-b0f3-9daccae62d5e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:57.307339 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:57.316632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0b023-0fa7-48bd-8435-eeb7b50f6ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:57.317268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b023-0fa7-48bd-8435-eeb7b50f6ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.317542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b023-0fa7-48bd-8435-eeb7b50f6ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.317812 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b023-0fa7-48bd-8435-eeb7b50f6ba8 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.319164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0b023-0fa7-48bd-8435-eeb7b50f6ba8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:57.319480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0b023-0fa7-48bd-8435-eeb7b50f6ba8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:57.321308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0b023-0fa7-48bd-8435-eeb7b50f6ba8 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:57.331673 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a4a51-5a86-4b16-81eb-cba66f9dad51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:57.333102 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a4a51-5a86-4b16-81eb-cba66f9dad51 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.333577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a4a51-5a86-4b16-81eb-cba66f9dad51 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.334055 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a4a51-5a86-4b16-81eb-cba66f9dad51 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.334996 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a4a51-5a86-4b16-81eb-cba66f9dad51 None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:57.335493 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a4a51-5a86-4b16-81eb-cba66f9dad51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:57.337271 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a4a51-5a86-4b16-81eb-cba66f9dad51 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:57.356094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 243/495] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:30:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:57.356094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a4a51-5a86-4b16-81eb-cba66f9dad51 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:57.372238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 253/496] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Aug 30 14:30:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:57.373483 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0b023-0fa7-48bd-8435-eeb7b50f6ba8 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:57.487716 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a4a51-5a86-4b16-81eb-cba66f9dad51 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:57.502047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0b023-0fa7-48bd-8435-eeb7b50f6ba8 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:57.509924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:57.533344 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c8481-c651-46ba-bf82-1c0ef0839d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:57.534332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c8481-c651-46ba-bf82-1c0ef0839d1f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.534738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c8481-c651-46ba-bf82-1c0ef0839d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.535188 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c8481-c651-46ba-bf82-1c0ef0839d1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.546161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4a437-fb27-4315-85cd-c71e2ce5ef6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:57.546894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4a437-fb27-4315-85cd-c71e2ce5ef6f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.547276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4a437-fb27-4315-85cd-c71e2ce5ef6f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.547558 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4a437-fb27-4315-85cd-c71e2ce5ef6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.551310 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c8481-c651-46ba-bf82-1c0ef0839d1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:57.553122 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c8481-c651-46ba-bf82-1c0ef0839d1f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:57.559711 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4a437-fb27-4315-85cd-c71e2ce5ef6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:57.561379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4a437-fb27-4315-85cd-c71e2ce5ef6f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:57.572118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-445c8481-c651-46ba-bf82-1c0ef0839d1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:57.572118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:57.572118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:57.572118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:57.572118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:57.572118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:57.572118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:57.572118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:57.572118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:57.572118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:57.572118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:57.572118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:57.572118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:57.572118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:57.572118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:57.572118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:57.573275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:57.573275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:57.573275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:57.573275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:57.573275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:57.573275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:57.573275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:57.573275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:57.573275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:57.573275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:57.573275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:57.573275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:57.573275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:57.573275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:57.573275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:57.573275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:57.573275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:57.574447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:57.574447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:57.574447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:57.574447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 244/497] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:57.576320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aace5dc-88fe-4215-ad76-56c1c3b0eefe None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.576776 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aace5dc-88fe-4215-ad76-56c1c3b0eefe None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.583059 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aace5dc-88fe-4215-ad76-56c1c3b0eefe None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.583059 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 245/498] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:57.583059 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aace5dc-88fe-4215-ad76-56c1c3b0eefe None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:57.594358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-5fe4a437-fb27-4315-85cd-c71e2ce5ef6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:57.594358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:57.594358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:57.594358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:57.594358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:57.594358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:57.594358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:57.594358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:57.594358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:57.594358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:57.594358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:57.594358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:57.594358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:57.594358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:57.594358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:57.594358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:57.596150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:57.596150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:57.596150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:57.596150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:57.596150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:57.596150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:57.596150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:57.596150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:57.596150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:57.596150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:57.596150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:57.596150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:57.596150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:57.596150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:57.596150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:57.596150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:57.596150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:57.597614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:57.597614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:57.597614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:57.597614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 254/499] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:57.599707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d5f05-49f1-44fe-bbfe-fec4687619bb None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.600229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d5f05-49f1-44fe-bbfe-fec4687619bb None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.600687 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d5f05-49f1-44fe-bbfe-fec4687619bb None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.604429 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 255/500] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:30:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:57.605876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d5f05-49f1-44fe-bbfe-fec4687619bb None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:57.667910 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:57.742031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aace5dc-88fe-4215-ad76-56c1c3b0eefe None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:57.742291 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d5f05-49f1-44fe-bbfe-fec4687619bb None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:57.778600 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e17288-6579-4e08-9ab3-ab77146c6a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:57.779210 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e17288-6579-4e08-9ab3-ab77146c6a27 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.779495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e17288-6579-4e08-9ab3-ab77146c6a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.779755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e17288-6579-4e08-9ab3-ab77146c6a27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.794336 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9627c12-4ece-4fd0-b387-be4b58a39ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:57.795085 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9627c12-4ece-4fd0-b387-be4b58a39ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.795367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9627c12-4ece-4fd0-b387-be4b58a39ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.795621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9627c12-4ece-4fd0-b387-be4b58a39ca7 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.796942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9627c12-4ece-4fd0-b387-be4b58a39ca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:57.797249 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9627c12-4ece-4fd0-b387-be4b58a39ca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:57.798712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9627c12-4ece-4fd0-b387-be4b58a39ca7 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:57.818166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 256/501] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:57.819547 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9be98-17f6-4f0b-9b76-d919852d2a7e None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.819920 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9be98-17f6-4f0b-9b76-d919852d2a7e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.820225 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9be98-17f6-4f0b-9b76-d919852d2a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.832382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:57.840632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-82c9be98-17f6-4f0b-9b76-d919852d2a7e None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:57.841856 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-82c9be98-17f6-4f0b-9b76-d919852d2a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:57.887821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e17288-6579-4e08-9ab3-ab77146c6a27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:57.889713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e17288-6579-4e08-9ab3-ab77146c6a27 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:57.924372 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 257/502] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:57.926151 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9be98-17f6-4f0b-9b76-d919852d2a7e None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:57.984839 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:58.036931 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 246/503] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:58.036931 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c916e9f-4b4d-4bd3-b4c8-fbeffcd7727d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:58.036931 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c916e9f-4b4d-4bd3-b4c8-fbeffcd7727d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:58.036931 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c916e9f-4b4d-4bd3-b4c8-fbeffcd7727d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:58.057755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2c916e9f-4b4d-4bd3-b4c8-fbeffcd7727d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:58.059006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2c916e9f-4b4d-4bd3-b4c8-fbeffcd7727d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:58.106538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9be98-17f6-4f0b-9b76-d919852d2a7e None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:58.147153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc6b6d-0595-4296-8df9-54624495d384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:58.147939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc6b6d-0595-4296-8df9-54624495d384 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:58.148330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc6b6d-0595-4296-8df9-54624495d384 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:58.148613 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc6b6d-0595-4296-8df9-54624495d384 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:58.149281 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc6b6d-0595-4296-8df9-54624495d384 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:58.149583 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc6b6d-0595-4296-8df9-54624495d384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:58.151027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc6b6d-0595-4296-8df9-54624495d384 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:58.164307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 258/504] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:58.169770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc6b6d-0595-4296-8df9-54624495d384 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:58.170524 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:58.180871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 247/505] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:58.182665 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c916e9f-4b4d-4bd3-b4c8-fbeffcd7727d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:58.327565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c916e9f-4b4d-4bd3-b4c8-fbeffcd7727d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:58.348972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:58.367440 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc6b6d-0595-4296-8df9-54624495d384 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:58.382144 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1903a-a964-4515-810f-66f45ea62068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:58.383528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1903a-a964-4515-810f-66f45ea62068 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:58.384202 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1903a-a964-4515-810f-66f45ea62068 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:58.384983 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1903a-a964-4515-810f-66f45ea62068 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:58.407677 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1903a-a964-4515-810f-66f45ea62068 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:58.411073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1903a-a964-4515-810f-66f45ea62068 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:58.433453 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d49fb5-7187-4616-ad43-a5920111e80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:58.434023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-15e1903a-a964-4515-810f-66f45ea62068 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:58.434023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:58.434023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:58.434023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:58.434023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:58.434023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:58.434023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:58.434023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:58.434023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:58.434023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:58.434023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:58.434023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:58.434023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:58.434023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:58.434023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:58.434023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:58.434953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:58.434953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:58.434953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:58.434953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:58.434953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:58.434953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:58.434953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:58.434953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:58.434953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:58.434953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:58.434953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:58.434953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:58.434953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:58.434953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:58.434953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:58.434953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:58.434953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:58.435788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:58.435788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:58.435788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:58.435951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d49fb5-7187-4616-ad43-a5920111e80c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:58.435951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d49fb5-7187-4616-ad43-a5920111e80c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:58.435951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d49fb5-7187-4616-ad43-a5920111e80c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:58.440907 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 248/506] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:58.443556 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1903a-a964-4515-810f-66f45ea62068 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:58.458594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d49fb5-7187-4616-ad43-a5920111e80c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:58.461561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d49fb5-7187-4616-ad43-a5920111e80c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:58.494348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-c8d49fb5-7187-4616-ad43-a5920111e80c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:30:58.494348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:58.494348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:58.494348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:58.494348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:58.494348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:58.494348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:58.494348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:58.494348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:58.494348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:58.494348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:58.494348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:58.494348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:58.494348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:58.494348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:58.494348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:58.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:58.496297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:58.496297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:58.496297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:58.496297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:58.496297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:58.496297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:58.496297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:58.496297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:58.496297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:30:58.496297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:58.496822 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 259/507] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:58.498039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d49fb5-7187-4616-ad43-a5920111e80c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:58.534436 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:58.624544 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1903a-a964-4515-810f-66f45ea62068 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:58.633327 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d49fb5-7187-4616-ad43-a5920111e80c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:58.669895 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba103a44-f647-48b7-a646-e02fee849650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:58.670713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba103a44-f647-48b7-a646-e02fee849650 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:58.671215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba103a44-f647-48b7-a646-e02fee849650 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:58.672365 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba103a44-f647-48b7-a646-e02fee849650 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:58.674512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a2cf1-ca5b-43f0-b2cc-d4eec6a731d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:58.675163 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a2cf1-ca5b-43f0-b2cc-d4eec6a731d4 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:58.675960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a2cf1-ca5b-43f0-b2cc-d4eec6a731d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:58.675960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a2cf1-ca5b-43f0-b2cc-d4eec6a731d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:58.681822 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba103a44-f647-48b7-a646-e02fee849650 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:58.683471 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba103a44-f647-48b7-a646-e02fee849650 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:58.688670 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a2cf1-ca5b-43f0-b2cc-d4eec6a731d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:58.690188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a2cf1-ca5b-43f0-b2cc-d4eec6a731d4 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:58.703626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-ba103a44-f647-48b7-a646-e02fee849650 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:30:58.703626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:58.703626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:58.703626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:58.703626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:58.703626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:58.703626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:58.703626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:58.703626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:58.703626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:58.703626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:58.703626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:58.703626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:58.703626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:58.703626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:58.703626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:58.705664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:58.707408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:58.707408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:58.707408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:58.707408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:58.707408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:58.707408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:58.707408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:58.707408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:58.707408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:30:58.707408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:30:58.707408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 249/508] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:58.708972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-a76a2cf1-ca5b-43f0-b2cc-d4eec6a731d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:58.708972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:58.708972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:58.708972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:58.708972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:58.708972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:58.708972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:58.708972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:58.708972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:58.708972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:58.708972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:58.708972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:58.708972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:58.708972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:58.708972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:58.708972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:58.710474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:58.710474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:58.710474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:58.710474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:58.710474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:58.710474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:58.710474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:58.710474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:58.710474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:58.710474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:58.710474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:58.710474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:58.710474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:58.710474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:58.710474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:58.710474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:58.710474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:58.713426 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:58.713426 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:58.713426 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:58.713854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba103a44-f647-48b7-a646-e02fee849650 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:58.713995 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 260/509] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:58.713995 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a2cf1-ca5b-43f0-b2cc-d4eec6a731d4 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:58.716518 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:58.877717 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba103a44-f647-48b7-a646-e02fee849650 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:58.883949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a2cf1-ca5b-43f0-b2cc-d4eec6a731d4 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:58.896396 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:58.923341 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d6787-9f9c-463e-876b-e9b29ff3f51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:58.924018 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d6787-9f9c-463e-876b-e9b29ff3f51f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:58.924304 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d6787-9f9c-463e-876b-e9b29ff3f51f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:58.924575 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d6787-9f9c-463e-876b-e9b29ff3f51f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:58.935321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366efbd9-45f5-449b-83ef-8c74b99fcb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:58.936076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366efbd9-45f5-449b-83ef-8c74b99fcb66 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:58.936353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366efbd9-45f5-449b-83ef-8c74b99fcb66 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:58.936748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366efbd9-45f5-449b-83ef-8c74b99fcb66 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:58.937428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366efbd9-45f5-449b-83ef-8c74b99fcb66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:58.937861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366efbd9-45f5-449b-83ef-8c74b99fcb66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:58.938012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d6787-9f9c-463e-876b-e9b29ff3f51f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:58.939295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366efbd9-45f5-449b-83ef-8c74b99fcb66 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:58.943395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-7f8d6787-9f9c-463e-876b-e9b29ff3f51f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:30:58.943395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:58.943395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:58.943395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:58.943395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:58.943395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:58.943395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:58.943395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:58.943395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:58.943395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:58.943395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:58.943395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:58.943395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:58.943395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:58.943395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:58.943395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:58.944578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:58.944578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:58.944578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:58.944578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:58.944578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:58.944578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:58.944578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:58.944578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:58.944578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:58.944578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:58.944578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:58.944578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:58.944578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:58.944578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:58.944578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:58.944578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:58.945376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:58.945376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:58.945376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:58.945376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:58.945376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:58.945376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:58.945376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:58.945376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:58.945376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:58.945376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:58.945376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:30:58.945376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:58.945376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 261/510] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:30:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:58.946040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d6787-9f9c-463e-876b-e9b29ff3f51f None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:58.957320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 250/511] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:58.959043 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366efbd9-45f5-449b-83ef-8c74b99fcb66 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:59.033431 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:59.076766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d6787-9f9c-463e-876b-e9b29ff3f51f None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:59.113002 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366efbd9-45f5-449b-83ef-8c74b99fcb66 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:59.121047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb859999-eda9-4c13-8c01-2f4e956893f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:59.121985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb859999-eda9-4c13-8c01-2f4e956893f2 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:59.122349 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb859999-eda9-4c13-8c01-2f4e956893f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:59.122625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb859999-eda9-4c13-8c01-2f4e956893f2 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:59.123462 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb859999-eda9-4c13-8c01-2f4e956893f2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:59.123875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb859999-eda9-4c13-8c01-2f4e956893f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:59.125536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb859999-eda9-4c13-8c01-2f4e956893f2 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:59.151198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 262/512] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Aug 30 14:30:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:59.151198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb859999-eda9-4c13-8c01-2f4e956893f2 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:59.165124 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990618b2-eb9c-4dd4-a2b1-76105568759b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:59.165911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990618b2-eb9c-4dd4-a2b1-76105568759b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:59.166294 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990618b2-eb9c-4dd4-a2b1-76105568759b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:59.166718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990618b2-eb9c-4dd4-a2b1-76105568759b None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:59.167539 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990618b2-eb9c-4dd4-a2b1-76105568759b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:59.167950 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990618b2-eb9c-4dd4-a2b1-76105568759b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:59.169585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990618b2-eb9c-4dd4-a2b1-76105568759b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:59.218959 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 251/513] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Aug 30 14:30:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:59.219972 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990618b2-eb9c-4dd4-a2b1-76105568759b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:59.251328 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:59.278289 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb859999-eda9-4c13-8c01-2f4e956893f2 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:59.302559 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:59.320814 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c149a-3c21-4de4-85fa-cad25cc140be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:59.321440 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c149a-3c21-4de4-85fa-cad25cc140be None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:59.321712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c149a-3c21-4de4-85fa-cad25cc140be None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:59.321979 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c149a-3c21-4de4-85fa-cad25cc140be None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:59.322650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c149a-3c21-4de4-85fa-cad25cc140be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:59.322945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c149a-3c21-4de4-85fa-cad25cc140be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:59.324387 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c149a-3c21-4de4-85fa-cad25cc140be None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:59.337668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 263/514] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:59.338773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c149a-3c21-4de4-85fa-cad25cc140be None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:59.358987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990618b2-eb9c-4dd4-a2b1-76105568759b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:59.412967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b3f2a-027f-4f15-8ca9-0eba131673b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:59.413511 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b3f2a-027f-4f15-8ca9-0eba131673b3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:59.413797 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b3f2a-027f-4f15-8ca9-0eba131673b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:59.414061 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b3f2a-027f-4f15-8ca9-0eba131673b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:59.414754 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b3f2a-027f-4f15-8ca9-0eba131673b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:59.415058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b3f2a-027f-4f15-8ca9-0eba131673b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:59.416549 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b3f2a-027f-4f15-8ca9-0eba131673b3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:59.425730 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 252/515] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:30:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:59.426610 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b3f2a-027f-4f15-8ca9-0eba131673b3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:59.458315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c149a-3c21-4de4-85fa-cad25cc140be None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:59.502854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d04fd5-dd74-472c-aa51-c84519b741c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:59.503530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d04fd5-dd74-472c-aa51-c84519b741c6 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:59.503810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d04fd5-dd74-472c-aa51-c84519b741c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:59.504117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d04fd5-dd74-472c-aa51-c84519b741c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:59.512302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d04fd5-dd74-472c-aa51-c84519b741c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:59.513857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d04fd5-dd74-472c-aa51-c84519b741c6 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:59.527969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-73d04fd5-dd74-472c-aa51-c84519b741c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:59.527969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:59.527969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:59.527969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:59.527969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:59.527969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:59.527969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:59.527969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:59.527969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:59.527969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:59.527969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:59.527969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:59.527969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:59.527969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:59.527969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:59.529046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:59.529046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:59.529046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:59.529046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:59.529046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:59.529046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:59.529046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:59.529046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:59.529046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:59.529046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:59.529046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:59.529046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:59.529046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:59.529046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:59.529046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:59.529046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:59.529046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:59.529935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:59.529935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:59.529935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:59.529935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:59.529935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:59.529935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:59.529935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:59.529935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:30:59.529935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 264/516] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:59.530470 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d04fd5-dd74-472c-aa51-c84519b741c6 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:59.536128 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:59.546242 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b3f2a-027f-4f15-8ca9-0eba131673b3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:59.590147 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:59.595702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b512a-81d0-444b-be90-75530d482e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:59.596762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b512a-81d0-444b-be90-75530d482e3e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:59.597136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b512a-81d0-444b-be90-75530d482e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:59.597523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b512a-81d0-444b-be90-75530d482e3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:59.667989 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d04fd5-dd74-472c-aa51-c84519b741c6 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:59.693541 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b512a-81d0-444b-be90-75530d482e3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:59.695055 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b512a-81d0-444b-be90-75530d482e3e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:59.711158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdc6b2-4cea-4b55-b0f4-23dbaf8c88a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:59.711797 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdc6b2-4cea-4b55-b0f4-23dbaf8c88a5 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:59.712107 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdc6b2-4cea-4b55-b0f4-23dbaf8c88a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:59.712379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdc6b2-4cea-4b55-b0f4-23dbaf8c88a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:59.791955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdc6b2-4cea-4b55-b0f4-23dbaf8c88a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:59.793452 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdc6b2-4cea-4b55-b0f4-23dbaf8c88a5 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:59.803060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:59.814984 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 253/517] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:59.815846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b512a-81d0-444b-be90-75530d482e3e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:59.844289 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:59.911891 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 265/518] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:59.912810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdc6b2-4cea-4b55-b0f4-23dbaf8c88a5 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:59.940517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b512a-81d0-444b-be90-75530d482e3e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:59.980811 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557b7b7-8e9c-49e8-b38c-a7ea245c6d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:59.981445 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557b7b7-8e9c-49e8-b38c-a7ea245c6d8d None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:59.981713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557b7b7-8e9c-49e8-b38c-a7ea245c6d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:59.981980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557b7b7-8e9c-49e8-b38c-a7ea245c6d8d None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:59.982640 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557b7b7-8e9c-49e8-b38c-a7ea245c6d8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:59.984490 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557b7b7-8e9c-49e8-b38c-a7ea245c6d8d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:00.009447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 254/519] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:30:59 2022] POST /identity/v3/users => generated 247 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:31:00.010861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557b7b7-8e9c-49e8-b38c-a7ea245c6d8d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:00.048014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdc6b2-4cea-4b55-b0f4-23dbaf8c88a5 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:00.094957 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:00.111551 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbf63a-90e3-413b-a3da-fce63b9a7a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:00.112577 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbf63a-90e3-413b-a3da-fce63b9a7a30 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:00.112998 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbf63a-90e3-413b-a3da-fce63b9a7a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:00.113425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbf63a-90e3-413b-a3da-fce63b9a7a30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:00.130176 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:00.134644 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbf63a-90e3-413b-a3da-fce63b9a7a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:00.137497 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbf63a-90e3-413b-a3da-fce63b9a7a30 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:00.165894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-dabbf63a-90e3-413b-a3da-fce63b9a7a30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:00.165894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:00.165894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:00.165894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:00.165894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:00.165894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:00.165894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:00.165894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:00.165894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:00.165894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:00.165894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:00.165894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:00.165894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:00.165894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:00.165894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:00.165894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:00.166969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:00.166969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:00.166969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:00.166969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:00.166969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:00.166969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:00.166969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:00.166969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:00.166969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:00.166969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:00.166969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:00.166969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:00.166969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:00.166969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:00.166969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:00.166969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:00.166969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:00.167951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:00.167951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:00.167951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:00.167951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 266/520] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:00.169171 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbf63a-90e3-413b-a3da-fce63b9a7a30 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:00.175728 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557b7b7-8e9c-49e8-b38c-a7ea245c6d8d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:00.216629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d8521-de85-4677-a36e-f69886ca071a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:00.217172 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d8521-de85-4677-a36e-f69886ca071a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:00.217470 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d8521-de85-4677-a36e-f69886ca071a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:00.217858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d8521-de85-4677-a36e-f69886ca071a None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:00.218652 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d8521-de85-4677-a36e-f69886ca071a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:00.219021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d8521-de85-4677-a36e-f69886ca071a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:00.220503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d8521-de85-4677-a36e-f69886ca071a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:00.244043 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 255/521] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:00.244043 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d8521-de85-4677-a36e-f69886ca071a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:00.348304 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbf63a-90e3-413b-a3da-fce63b9a7a30 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:00.398008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d8521-de85-4677-a36e-f69886ca071a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:00.407943 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:00.411795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9267b7b-7ffb-4aa8-9db8-4a95afd87ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:00.413520 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9267b7b-7ffb-4aa8-9db8-4a95afd87ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:00.414253 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9267b7b-7ffb-4aa8-9db8-4a95afd87ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:00.414940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9267b7b-7ffb-4aa8-9db8-4a95afd87ff1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:00.435506 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9267b7b-7ffb-4aa8-9db8-4a95afd87ff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:00.437818 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9267b7b-7ffb-4aa8-9db8-4a95afd87ff1 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:00.441947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a34417-6d75-4d76-a62a-e90cc96b0709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:00.442932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a34417-6d75-4d76-a62a-e90cc96b0709 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:00.443209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a34417-6d75-4d76-a62a-e90cc96b0709 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:00.443474 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a34417-6d75-4d76-a62a-e90cc96b0709 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:00.455537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a34417-6d75-4d76-a62a-e90cc96b0709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:00.457115 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a34417-6d75-4d76-a62a-e90cc96b0709 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:00.462846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-f9267b7b-7ffb-4aa8-9db8-4a95afd87ff1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:00.462846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:00.462846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:00.462846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:00.462846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:00.462846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:00.462846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:00.462846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:00.462846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:00.462846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:00.462846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:00.462846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:00.462846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:00.462846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:00.462846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:00.462846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:00.463983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:00.463983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:00.463983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:00.463983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:00.463983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:00.463983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:00.463983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:00.463983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:00.463983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:00.463983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:00.463983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:00.463983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:00.463983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:00.463983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:00.463983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:00.463983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:00.463983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:00.465268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:00.465268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:00.465268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:00.472524 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 267/522] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:00.473908 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9267b7b-7ffb-4aa8-9db8-4a95afd87ff1 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:00.475662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-f4a34417-6d75-4d76-a62a-e90cc96b0709 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:00.475662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:00.475662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:00.475662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:00.475662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:00.475662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:00.475662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:00.475662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:00.475662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:00.475662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:00.475662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:00.475662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:00.475662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:00.475662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:00.475662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:00.475662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:00.476793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:00.476793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:00.476793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:00.476793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:00.476793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:00.476793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:00.476793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:00.476793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:00.476793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:00.476793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:00.476793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:00.476793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:00.476793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:00.476793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:00.476793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:00.476793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:00.476793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:00.477666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:00.477666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:00.477666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:00.477666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 256/523] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:00.478277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a34417-6d75-4d76-a62a-e90cc96b0709 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:00.543316 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:00.621524 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9267b7b-7ffb-4aa8-9db8-4a95afd87ff1 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:00.621789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a34417-6d75-4d76-a62a-e90cc96b0709 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:00.673950 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848a0a5-5e20-4c77-a6bd-24ccc6e6e209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:00.674911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848a0a5-5e20-4c77-a6bd-24ccc6e6e209 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:00.675361 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848a0a5-5e20-4c77-a6bd-24ccc6e6e209 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:00.675809 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848a0a5-5e20-4c77-a6bd-24ccc6e6e209 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:00.677152 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848a0a5-5e20-4c77-a6bd-24ccc6e6e209 None admin] RBAC: Adding query filter params (role.id=f0fd911d7c9246aeb3d67299473a9f42, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=cc95bbdafb0c4f34ad0662bc2e3ba4ec) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:00.677666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848a0a5-5e20-4c77-a6bd-24ccc6e6e209 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:00.679073 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb329b-3a35-421a-b014-13bcaf54682b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:00.680068 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb329b-3a35-421a-b014-13bcaf54682b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:00.680260 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848a0a5-5e20-4c77-a6bd-24ccc6e6e209 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:00.680617 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb329b-3a35-421a-b014-13bcaf54682b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:00.681036 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb329b-3a35-421a-b014-13bcaf54682b None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:00.682155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb329b-3a35-421a-b014-13bcaf54682b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:00.682615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb329b-3a35-421a-b014-13bcaf54682b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:00.684974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb329b-3a35-421a-b014-13bcaf54682b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:00.696046 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 257/524] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:00 2022] GET /identity/v3/role_assignments?user.id=cc95bbdafb0c4f34ad0662bc2e3ba4ec&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=f0fd911d7c9246aeb3d67299473a9f42 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:00.696046 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848a0a5-5e20-4c77-a6bd-24ccc6e6e209 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:00.703448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 268/525] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:00.705321 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb329b-3a35-421a-b014-13bcaf54682b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:00.756415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:00.886829 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848a0a5-5e20-4c77-a6bd-24ccc6e6e209 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:00.907751 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb329b-3a35-421a-b014-13bcaf54682b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:00.912478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:00.942428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad52635-7661-40bb-bff3-191f808173f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:00.943066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad52635-7661-40bb-bff3-191f808173f1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:00.943653 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad52635-7661-40bb-bff3-191f808173f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:00.943653 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad52635-7661-40bb-bff3-191f808173f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:00.944513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad52635-7661-40bb-bff3-191f808173f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:00.946260 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad52635-7661-40bb-bff3-191f808173f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:00.949652 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad52635-7661-40bb-bff3-191f808173f1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:00.962874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104abb9e-3ff9-4623-bcfc-d6a221186ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:00.963492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104abb9e-3ff9-4623-bcfc-d6a221186ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:00.963767 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104abb9e-3ff9-4623-bcfc-d6a221186ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:00.964063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104abb9e-3ff9-4623-bcfc-d6a221186ee3 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:00.964732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104abb9e-3ff9-4623-bcfc-d6a221186ee3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:00.965034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104abb9e-3ff9-4623-bcfc-d6a221186ee3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:00.966456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104abb9e-3ff9-4623-bcfc-d6a221186ee3 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:00.985857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 269/526] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:00.986901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104abb9e-3ff9-4623-bcfc-d6a221186ee3 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:00.992380 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 258/527] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:00.994157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad52635-7661-40bb-bff3-191f808173f1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:01.160444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:01.197656 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad52635-7661-40bb-bff3-191f808173f1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:01.206452 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104abb9e-3ff9-4623-bcfc-d6a221186ee3 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:01.256598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1be47-9407-4517-82b6-c87fb09a8830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:01.257614 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1be47-9407-4517-82b6-c87fb09a8830 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:01.258068 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1be47-9407-4517-82b6-c87fb09a8830 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:01.258524 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1be47-9407-4517-82b6-c87fb09a8830 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:01.270661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1be47-9407-4517-82b6-c87fb09a8830 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:01.273489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1be47-9407-4517-82b6-c87fb09a8830 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:01.276021 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f6426-dcf6-4100-a5a0-b26ccf6f8d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:01.277466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f6426-dcf6-4100-a5a0-b26ccf6f8d71 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:01.277917 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f6426-dcf6-4100-a5a0-b26ccf6f8d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:01.278936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f6426-dcf6-4100-a5a0-b26ccf6f8d71 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:01.294770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f6426-dcf6-4100-a5a0-b26ccf6f8d71 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:01.297233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-5eb1be47-9407-4517-82b6-c87fb09a8830 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:01.297233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:01.297233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:01.297233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:01.297233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:01.297233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:01.297233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:01.297233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:01.297233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:01.297233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:01.297233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:01.297233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:01.297233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:01.297233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:01.297233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:01.298237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:01.298237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:01.298237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:01.298237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:01.298237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:01.298237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:01.298237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:01.298237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:01.298237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:01.298237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:01.298237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:01.298237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:01.298237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:01.298237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:01.298237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:01.298237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:01.298237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:01.299286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:01.299286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:01.299286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:01.299286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:01.299286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:01.299286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:01.299286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:01.299286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:01.299786 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f6426-dcf6-4100-a5a0-b26ccf6f8d71 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:01.308067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 259/528] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:01.309652 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1be47-9407-4517-82b6-c87fb09a8830 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:01.321345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-1e0f6426-dcf6-4100-a5a0-b26ccf6f8d71 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:31:01.321345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:01.321345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:01.321345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:01.321345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:01.321345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:01.321345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:01.321345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:01.321345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:01.321345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:01.321345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:01.321345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:01.321345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:01.321345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:01.321345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:01.321345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:01.323312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:01.324449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:01.324449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:01.324449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:01.324449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:01.324449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:01.324449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:01.324449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:01.324449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:01.324449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:31:01.324449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:01.324449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 270/529] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:31:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:01.326045 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f6426-dcf6-4100-a5a0-b26ccf6f8d71 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:01.377250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:01.519356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1be47-9407-4517-82b6-c87fb09a8830 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:01.528654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f6426-dcf6-4100-a5a0-b26ccf6f8d71 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:01.595479 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9f117-01e7-4f98-9d50-4b2daf3b1104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:01.596854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9f117-01e7-4f98-9d50-4b2daf3b1104 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:01.597455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9f117-01e7-4f98-9d50-4b2daf3b1104 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:01.598063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9f117-01e7-4f98-9d50-4b2daf3b1104 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:01.601741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccccd8-34d3-4554-853a-c675d0ea54b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:01.603586 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccccd8-34d3-4554-853a-c675d0ea54b8 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:01.604291 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccccd8-34d3-4554-853a-c675d0ea54b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:01.604931 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccccd8-34d3-4554-853a-c675d0ea54b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:01.606319 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ccccd8-34d3-4554-853a-c675d0ea54b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:01.607009 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ccccd8-34d3-4554-853a-c675d0ea54b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:01.610158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ccccd8-34d3-4554-853a-c675d0ea54b8 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:01.611727 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9f117-01e7-4f98-9d50-4b2daf3b1104 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:01.614590 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9f117-01e7-4f98-9d50-4b2daf3b1104 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:01.644492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-f0b9f117-01e7-4f98-9d50-4b2daf3b1104 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:01.644492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:01.644492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:01.644492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:01.644492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:01.644492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:01.644492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:01.644492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:01.644492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:01.644492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:01.644492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:01.644492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:01.644492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:01.644492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:01.644492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:01.646387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:01.646387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:01.646387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:01.646387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:01.646387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:01.646387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:01.646387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:01.646387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:01.646387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:01.646387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:01.646387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:01.646387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:01.646387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:01.646387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:01.646387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:01.646387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:01.646387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:01.648513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:01.648513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:01.648513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:01.648513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:01.648513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:01.648513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:01.648513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:01.648513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:01.656179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 260/530] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:01.656179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9f117-01e7-4f98-9d50-4b2daf3b1104 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:01.659018 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 271/531] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:01.660796 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccccd8-34d3-4554-853a-c675d0ea54b8 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:01.665964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:01.830283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:01.869729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccccd8-34d3-4554-853a-c675d0ea54b8 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:01.884321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9f117-01e7-4f98-9d50-4b2daf3b1104 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:01.947361 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b8eba-1aa9-4c3e-ad3e-530130e3a295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:01.948981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b8eba-1aa9-4c3e-ad3e-530130e3a295 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:01.950192 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b8eba-1aa9-4c3e-ad3e-530130e3a295 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:01.950843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b8eba-1aa9-4c3e-ad3e-530130e3a295 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:01.952824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b8eba-1aa9-4c3e-ad3e-530130e3a295 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:01.953694 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b8eba-1aa9-4c3e-ad3e-530130e3a295 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:01.957070 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b8eba-1aa9-4c3e-ad3e-530130e3a295 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:01.958092 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6956d-0fc4-4c64-99db-d4e65219584a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:01.959181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6956d-0fc4-4c64-99db-d4e65219584a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:01.959739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6956d-0fc4-4c64-99db-d4e65219584a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:01.960425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6956d-0fc4-4c64-99db-d4e65219584a None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:01.961760 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6956d-0fc4-4c64-99db-d4e65219584a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:01.962372 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6956d-0fc4-4c64-99db-d4e65219584a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:01.965058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6956d-0fc4-4c64-99db-d4e65219584a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:01.992124 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 272/532] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:01.996165 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b8eba-1aa9-4c3e-ad3e-530130e3a295 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:01.998690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 261/533] 10.222.0.25 () {56 vars in 1139 bytes} [Tue Aug 30 14:31:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:02.002676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6956d-0fc4-4c64-99db-d4e65219584a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:02.089138 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:02.164532 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6956d-0fc4-4c64-99db-d4e65219584a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:02.199839 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b8eba-1aa9-4c3e-ad3e-530130e3a295 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:02.210152 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0a218-502e-40ed-933b-71a38b5410b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:02.210782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0a218-502e-40ed-933b-71a38b5410b3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:02.211065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0a218-502e-40ed-933b-71a38b5410b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:02.211333 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0a218-502e-40ed-933b-71a38b5410b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:02.212118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0a218-502e-40ed-933b-71a38b5410b3 None admin] RBAC: Adding query filter params (role.id=f0fd911d7c9246aeb3d67299473a9f42, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=2788fece401a4a7ebdb516afe2dd8ec8) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:02.212428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0a218-502e-40ed-933b-71a38b5410b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:02.213962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0a218-502e-40ed-933b-71a38b5410b3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:02.220479 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 262/534] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:01 2022] GET /identity/v3/role_assignments?user.id=2788fece401a4a7ebdb516afe2dd8ec8&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=f0fd911d7c9246aeb3d67299473a9f42 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:02.221391 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0a218-502e-40ed-933b-71a38b5410b3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:02.255673 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e7c89-e26d-4fc6-8ef6-aaeb5d5f5bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:02.257160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e7c89-e26d-4fc6-8ef6-aaeb5d5f5bdb None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:02.257740 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e7c89-e26d-4fc6-8ef6-aaeb5d5f5bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:02.258201 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e7c89-e26d-4fc6-8ef6-aaeb5d5f5bdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:02.259267 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e7c89-e26d-4fc6-8ef6-aaeb5d5f5bdb None admin] RBAC: Adding query filter params (role.id=f0fd911d7c9246aeb3d67299473a9f42, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=69de9368993f402cb6325e5022e064d4) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:02.259818 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e7c89-e26d-4fc6-8ef6-aaeb5d5f5bdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:02.261753 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e7c89-e26d-4fc6-8ef6-aaeb5d5f5bdb None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:02.269345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 273/535] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:01 2022] GET /identity/v3/role_assignments?user.id=69de9368993f402cb6325e5022e064d4&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=f0fd911d7c9246aeb3d67299473a9f42 => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:02.271023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5175c-51cb-4171-a00f-784f63a710e1 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:02.271520 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5175c-51cb-4171-a00f-784f63a710e1 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:02.272020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5175c-51cb-4171-a00f-784f63a710e1 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:02.274064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 274/536] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:02.279724 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b139b6-ce6f-41c3-8838-ffe8d70571f6 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:02.280455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b139b6-ce6f-41c3-8838-ffe8d70571f6 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:02.284770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b139b6-ce6f-41c3-8838-ffe8d70571f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:02.326406 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-69b139b6-ce6f-41c3-8838-ffe8d70571f6 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:02.329351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-69b139b6-ce6f-41c3-8838-ffe8d70571f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:02.334930 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:02.513430 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0a218-502e-40ed-933b-71a38b5410b3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:02.550019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 275/537] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:31:02.567963 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:02.592625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98fb5b-de05-4ff8-8d6c-7e451294e11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:02.592625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98fb5b-de05-4ff8-8d6c-7e451294e11b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:02.592625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98fb5b-de05-4ff8-8d6c-7e451294e11b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:02.592625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98fb5b-de05-4ff8-8d6c-7e451294e11b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:02.719987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98fb5b-de05-4ff8-8d6c-7e451294e11b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:02.721504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98fb5b-de05-4ff8-8d6c-7e451294e11b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:02.839407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 263/538] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:31:02.845200 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b139b6-ce6f-41c3-8838-ffe8d70571f6 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:02.889522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d424c8b-1619-43e9-866c-735d8e813b04 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:02.889955 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d424c8b-1619-43e9-866c-735d8e813b04 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:02.890262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d424c8b-1619-43e9-866c-735d8e813b04 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:02.893396 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 264/539] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:02.898633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88699293-58ad-4e0e-9267-0a2dd9f54b2b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:02.899047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88699293-58ad-4e0e-9267-0a2dd9f54b2b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:02.899243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88699293-58ad-4e0e-9267-0a2dd9f54b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:02.920035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-88699293-58ad-4e0e-9267-0a2dd9f54b2b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:02.921340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-88699293-58ad-4e0e-9267-0a2dd9f54b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:02.946942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:02.994608 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b139b6-ce6f-41c3-8838-ffe8d70571f6 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:03.007654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 265/540] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:31:03.008775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88699293-58ad-4e0e-9267-0a2dd9f54b2b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:03.036399 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d698dc-d67a-40b4-95a5-478a3fff1e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:03.037050 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d698dc-d67a-40b4-95a5-478a3fff1e6b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:03.037326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d698dc-d67a-40b4-95a5-478a3fff1e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:03.038129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d698dc-d67a-40b4-95a5-478a3fff1e6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:03.050969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d698dc-d67a-40b4-95a5-478a3fff1e6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:03.052695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d698dc-d67a-40b4-95a5-478a3fff1e6b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:03.064488 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:03.073409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-36d698dc-d67a-40b4-95a5-478a3fff1e6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:03.073409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:03.073409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:03.073409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:03.073409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:03.073409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:03.073409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:03.073409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:03.073409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:03.073409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:03.073409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:03.073409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:03.073409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:03.073409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:03.073409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:03.073409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:03.076475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:03.076475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:03.076475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:03.076475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:03.076475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:03.076475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:03.076475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:03.076475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:03.076475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:03.076475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:03.076475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:03.076475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:03.076475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:03.076475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:03.076475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:03.076475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:03.076475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:03.078239 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:03.078239 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:03.078239 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:03.078239 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 276/541] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:31:03.087304 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d698dc-d67a-40b4-95a5-478a3fff1e6b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:03.213026 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88699293-58ad-4e0e-9267-0a2dd9f54b2b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:03.293296 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab59c71-4899-4cc8-96fd-66de00329c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:03.293947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab59c71-4899-4cc8-96fd-66de00329c66 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:03.294238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab59c71-4899-4cc8-96fd-66de00329c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:03.294503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab59c71-4899-4cc8-96fd-66de00329c66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:03.319334 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab59c71-4899-4cc8-96fd-66de00329c66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:03.321184 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d698dc-d67a-40b4-95a5-478a3fff1e6b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:03.327211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-9ab59c71-4899-4cc8-96fd-66de00329c66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:03.327211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:03.327211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:03.327211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:03.327211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:03.327211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:03.327211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:03.327211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:03.327211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:03.327211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:03.327211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:03.327211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:03.327211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:03.327211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:03.327211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:03.327211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:03.330528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:03.330528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:03.330528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:03.330528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:03.330528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:03.330528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:03.330528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:03.330528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:03.330528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:03.330528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:03.330528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:03.330528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:03.330528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:03.330528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:03.330528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:03.330528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:03.333980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:03.333980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:03.333980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:03.333980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:03.333980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:03.333980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:03.333980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:03.333980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:03.333980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:03.333980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:03.333980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:03.333980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:03.336280 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 266/542] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:31:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:03.336280 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab59c71-4899-4cc8-96fd-66de00329c66 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:03.357613 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:03.372834 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd8721-cade-43af-979e-63c9ec70fb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:03.373461 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd8721-cade-43af-979e-63c9ec70fb83 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:03.373740 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd8721-cade-43af-979e-63c9ec70fb83 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:03.374256 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd8721-cade-43af-979e-63c9ec70fb83 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:03.375034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd8721-cade-43af-979e-63c9ec70fb83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:03.375436 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd8721-cade-43af-979e-63c9ec70fb83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:03.376914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd8721-cade-43af-979e-63c9ec70fb83 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:03.391433 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 277/543] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:03.392367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd8721-cade-43af-979e-63c9ec70fb83 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:03.453499 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:03.474191 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab59c71-4899-4cc8-96fd-66de00329c66 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:03.510633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12453cc1-de93-4cb4-81ad-7d5b7319c727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:03.511258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12453cc1-de93-4cb4-81ad-7d5b7319c727 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:03.511564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12453cc1-de93-4cb4-81ad-7d5b7319c727 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:03.511904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12453cc1-de93-4cb4-81ad-7d5b7319c727 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:03.512630 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12453cc1-de93-4cb4-81ad-7d5b7319c727 None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:03.512932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12453cc1-de93-4cb4-81ad-7d5b7319c727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:03.514442 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12453cc1-de93-4cb4-81ad-7d5b7319c727 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:03.523859 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 267/544] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:31:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:03.524745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12453cc1-de93-4cb4-81ad-7d5b7319c727 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:03.556025 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd8721-cade-43af-979e-63c9ec70fb83 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:03.622213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b2824-0a03-4f51-8139-d7f186c2994e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:03.622435 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:03.623308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b2824-0a03-4f51-8139-d7f186c2994e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:03.623800 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b2824-0a03-4f51-8139-d7f186c2994e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:03.624482 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b2824-0a03-4f51-8139-d7f186c2994e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:03.636488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12453cc1-de93-4cb4-81ad-7d5b7319c727 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:03.659560 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b2824-0a03-4f51-8139-d7f186c2994e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:03.673503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-e59b2824-0a03-4f51-8139-d7f186c2994e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:03.673503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:03.673503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:03.673503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:03.673503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:03.673503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:03.673503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:03.673503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:03.673503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:03.673503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:03.673503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:03.673503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:03.673503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:03.673503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:03.673503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:03.673503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:03.674492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:03.674492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:03.674492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:03.674492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:03.674492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:03.674492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:03.674492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:03.674492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:03.674492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:03.674492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:03.674492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:03.674492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:03.674492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:03.674492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:03.674492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:03.674492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:03.675296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:03.675296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:03.675296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:03.675296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:03.675296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:03.675296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:03.675296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:03.675296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:03.675296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:03.675296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:03.675296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:03.675296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:03.676210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 278/545] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:31:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:03.678802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b2824-0a03-4f51-8139-d7f186c2994e None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:03.688808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e170b-ca77-48d3-9f6f-e751127abf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:03.689887 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e170b-ca77-48d3-9f6f-e751127abf68 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:03.690397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e170b-ca77-48d3-9f6f-e751127abf68 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:03.690925 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e170b-ca77-48d3-9f6f-e751127abf68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:03.699344 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e170b-ca77-48d3-9f6f-e751127abf68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:03.701230 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e170b-ca77-48d3-9f6f-e751127abf68 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:03.715407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-778e170b-ca77-48d3-9f6f-e751127abf68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:03.715407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:03.715407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:03.715407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:03.715407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:03.715407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:03.715407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:03.715407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:03.715407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:03.715407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:03.715407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:03.715407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:03.715407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:03.715407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:03.715407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:03.716381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:03.716381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:03.716381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:03.716381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:03.716381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:03.716381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:03.716381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:03.716381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:03.716381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:03.716381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:03.716381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:03.716381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:03.716381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:03.716381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:03.716381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:03.716381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:03.716381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:03.717227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:03.717227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:03.717227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:03.717227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:03.717227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:03.717227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:03.717227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:03.717227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:03.724710 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 268/546] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:03.726128 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e170b-ca77-48d3-9f6f-e751127abf68 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:03.849475 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e170b-ca77-48d3-9f6f-e751127abf68 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:03.862265 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b2824-0a03-4f51-8139-d7f186c2994e None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:03.876368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:03.896917 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bf9ce-6399-456f-a573-fb18402d1ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:03.897536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bf9ce-6399-456f-a573-fb18402d1ebe None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:03.897816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bf9ce-6399-456f-a573-fb18402d1ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:03.898091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bf9ce-6399-456f-a573-fb18402d1ebe None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:03.898775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bf9ce-6399-456f-a573-fb18402d1ebe None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:03.899357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bf9ce-6399-456f-a573-fb18402d1ebe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:03.901462 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bf9ce-6399-456f-a573-fb18402d1ebe None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:03.905382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:03.911416 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13357c29-41ca-4b69-abf3-2bbd86a8984d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:03.912587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13357c29-41ca-4b69-abf3-2bbd86a8984d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:03.913062 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13357c29-41ca-4b69-abf3-2bbd86a8984d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:03.913520 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13357c29-41ca-4b69-abf3-2bbd86a8984d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:03.914267 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 269/547] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:31:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:03.915154 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bf9ce-6399-456f-a573-fb18402d1ebe None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:03.927343 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13357c29-41ca-4b69-abf3-2bbd86a8984d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:03.929846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13357c29-41ca-4b69-abf3-2bbd86a8984d None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:03.947901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-13357c29-41ca-4b69-abf3-2bbd86a8984d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:03.947901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:03.947901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:03.947901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:03.947901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:03.947901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:03.947901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:03.947901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:03.947901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:03.947901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:03.947901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:03.947901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:03.947901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:03.947901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:03.947901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:03.947901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:03.949082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:03.949082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:03.949082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:03.949082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:03.949082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:03.949082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:03.949082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:03.949082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:03.949082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:03.949082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:03.949082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:03.949082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:03.949082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:03.949082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:03.949082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:03.949082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:03.949082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:03.949933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:03.949933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:03.949933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:03.952477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 279/548] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:03.953790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13357c29-41ca-4b69-abf3-2bbd86a8984d None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:04.038886 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bf9ce-6399-456f-a573-fb18402d1ebe None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:04.076553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340aa678-e3ee-413e-a2f5-c0b6ed35a602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:04.077616 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340aa678-e3ee-413e-a2f5-c0b6ed35a602 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:04.078091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340aa678-e3ee-413e-a2f5-c0b6ed35a602 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:04.078581 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340aa678-e3ee-413e-a2f5-c0b6ed35a602 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:04.079493 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340aa678-e3ee-413e-a2f5-c0b6ed35a602 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:04.080039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340aa678-e3ee-413e-a2f5-c0b6ed35a602 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:04.081719 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340aa678-e3ee-413e-a2f5-c0b6ed35a602 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:04.100397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13357c29-41ca-4b69-abf3-2bbd86a8984d None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:04.100945 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 270/549] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:04.102137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340aa678-e3ee-413e-a2f5-c0b6ed35a602 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:04.145717 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cb88b-68b6-4b19-9cf0-077db9c8a4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:04.146372 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cb88b-68b6-4b19-9cf0-077db9c8a4ad None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:04.146649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cb88b-68b6-4b19-9cf0-077db9c8a4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:04.146920 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cb88b-68b6-4b19-9cf0-077db9c8a4ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:04.163400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399cb88b-68b6-4b19-9cf0-077db9c8a4ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:04.165355 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399cb88b-68b6-4b19-9cf0-077db9c8a4ad None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:04.170178 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:04.188680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-399cb88b-68b6-4b19-9cf0-077db9c8a4ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:04.188680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:04.188680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:04.188680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:04.188680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:04.188680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:04.188680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:04.188680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:04.188680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:04.188680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:04.188680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:04.188680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:04.188680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:04.188680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:04.188680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:04.188680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:04.189788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:04.189788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:04.189788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:04.189788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:04.189788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:04.189788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:04.189788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:04.189788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:04.189788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:04.189788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:04.189788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:04.189788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:04.189788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:04.189788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:04.189788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:04.189788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:04.189788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:04.191374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:04.191374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:04.191374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:04.191374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 280/550] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:04.191374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cb88b-68b6-4b19-9cf0-077db9c8a4ad None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:04.220629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:04.226572 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340aa678-e3ee-413e-a2f5-c0b6ed35a602 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:04.266746 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d5758-c77c-4cf3-917d-7b245e96bba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:04.267750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d5758-c77c-4cf3-917d-7b245e96bba1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:04.268258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d5758-c77c-4cf3-917d-7b245e96bba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:04.268708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d5758-c77c-4cf3-917d-7b245e96bba1 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:04.269595 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d5758-c77c-4cf3-917d-7b245e96bba1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:04.270102 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d5758-c77c-4cf3-917d-7b245e96bba1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:04.271725 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d5758-c77c-4cf3-917d-7b245e96bba1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:04.287089 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 271/551] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:04.288456 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d5758-c77c-4cf3-917d-7b245e96bba1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:04.345910 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cb88b-68b6-4b19-9cf0-077db9c8a4ad None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:04.385173 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba1bc7-5161-4226-bf9b-c48dc0e851a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:04.385800 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba1bc7-5161-4226-bf9b-c48dc0e851a1 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:04.386131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba1bc7-5161-4226-bf9b-c48dc0e851a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:04.386402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba1bc7-5161-4226-bf9b-c48dc0e851a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:04.387160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba1bc7-5161-4226-bf9b-c48dc0e851a1 None admin] RBAC: Adding query filter params (role.id=f0fd911d7c9246aeb3d67299473a9f42, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=932fce741c5a4deba79a310a14e6bd2c) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:04.387529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba1bc7-5161-4226-bf9b-c48dc0e851a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:04.389284 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba1bc7-5161-4226-bf9b-c48dc0e851a1 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:04.400107 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 281/552] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:04 2022] GET /identity/v3/role_assignments?user.id=932fce741c5a4deba79a310a14e6bd2c&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=f0fd911d7c9246aeb3d67299473a9f42 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:04.400107 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba1bc7-5161-4226-bf9b-c48dc0e851a1 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:04.432378 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d5758-c77c-4cf3-917d-7b245e96bba1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:04.462928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:04.477799 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784407c-c293-4c15-8b28-0fb365bfd5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:04.479016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784407c-c293-4c15-8b28-0fb365bfd5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:04.479532 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784407c-c293-4c15-8b28-0fb365bfd5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:04.480109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784407c-c293-4c15-8b28-0fb365bfd5d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:04.481031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784407c-c293-4c15-8b28-0fb365bfd5d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:04.481538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784407c-c293-4c15-8b28-0fb365bfd5d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:04.483434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784407c-c293-4c15-8b28-0fb365bfd5d8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:04.498815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 272/553] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:04.500673 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:04.502355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f251108-2526-4a96-a907-a89f1d1c388c None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:04.503127 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f251108-2526-4a96-a907-a89f1d1c388c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:04.503658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f251108-2526-4a96-a907-a89f1d1c388c None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:04.505466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 273/554] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:04.506328 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f251108-2526-4a96-a907-a89f1d1c388c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:04.534805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba1bc7-5161-4226-bf9b-c48dc0e851a1 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:04.577899 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e5761-46c0-409a-8b74-7bf472a9d7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:04.578578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e5761-46c0-409a-8b74-7bf472a9d7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:04.578860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e5761-46c0-409a-8b74-7bf472a9d7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:04.579205 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e5761-46c0-409a-8b74-7bf472a9d7a9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:04.589890 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e5761-46c0-409a-8b74-7bf472a9d7a9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:04.591566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e5761-46c0-409a-8b74-7bf472a9d7a9 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:04.605754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-785e5761-46c0-409a-8b74-7bf472a9d7a9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:31:04.605754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:04.605754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:04.605754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:04.605754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:04.605754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:04.605754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:04.605754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:04.605754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:04.605754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:04.605754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:04.605754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:04.605754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:04.605754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:04.605754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:04.605754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:04.606713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:04.607625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:04.607625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:04.607625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:04.607625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:04.607625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:04.607625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:04.607625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:04.607625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:04.607625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:31:04.607625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:04.612113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 282/555] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:04.613424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257bc52-a556-4df5-8474-2137a13ee72c None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:04.613711 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257bc52-a556-4df5-8474-2137a13ee72c None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:04.613986 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257bc52-a556-4df5-8474-2137a13ee72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:04.637291 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-7257bc52-a556-4df5-8474-2137a13ee72c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:04.638663 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7257bc52-a556-4df5-8474-2137a13ee72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:04.650353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f251108-2526-4a96-a907-a89f1d1c388c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:04.717132 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b07c72-62e5-4626-a3df-2b9e3c4a5dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:04.717904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 283/556] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:31:04.718276 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b07c72-62e5-4626-a3df-2b9e3c4a5dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:04.718762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b07c72-62e5-4626-a3df-2b9e3c4a5dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:04.718932 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257bc52-a556-4df5-8474-2137a13ee72c None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:04.719245 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b07c72-62e5-4626-a3df-2b9e3c4a5dd1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:04.741995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b07c72-62e5-4626-a3df-2b9e3c4a5dd1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:04.741995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b07c72-62e5-4626-a3df-2b9e3c4a5dd1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:04.784662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-92b07c72-62e5-4626-a3df-2b9e3c4a5dd1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:31:04.784662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:04.784662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:04.784662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:04.784662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:04.784662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:04.784662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:04.784662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:04.784662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:04.784662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:04.784662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:04.784662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:04.784662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:04.784662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:04.784662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:04.784662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:04.786147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:04.787110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:04.787110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:04.787110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:04.787110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:04.787110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:04.787110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:04.787110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:04.787110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:04.787110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:31:04.787110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:04.787657 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 274/557] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Aug 30 14:31:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:04.796030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:04.798028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b07c72-62e5-4626-a3df-2b9e3c4a5dd1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:04.835072 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257bc52-a556-4df5-8474-2137a13ee72c None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:04.873718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaad6ef-664d-4571-8f46-4b4568128ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:04.874424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaad6ef-664d-4571-8f46-4b4568128ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:04.874704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaad6ef-664d-4571-8f46-4b4568128ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:04.874979 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaad6ef-664d-4571-8f46-4b4568128ba9 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:04.875868 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaad6ef-664d-4571-8f46-4b4568128ba9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:04.876210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaad6ef-664d-4571-8f46-4b4568128ba9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:04.877737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaad6ef-664d-4571-8f46-4b4568128ba9 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:04.899178 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 284/558] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Aug 30 14:31:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:04.905163 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaad6ef-664d-4571-8f46-4b4568128ba9 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:04.946383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:05.047124 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaad6ef-664d-4571-8f46-4b4568128ba9 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:05.076204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b07c72-62e5-4626-a3df-2b9e3c4a5dd1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:05.100472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:05.117167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936ee76-4ded-4ac0-94ae-fc9b901745ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:05.117812 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936ee76-4ded-4ac0-94ae-fc9b901745ad None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:05.118092 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936ee76-4ded-4ac0-94ae-fc9b901745ad None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:05.118366 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936ee76-4ded-4ac0-94ae-fc9b901745ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:05.136132 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e6996-28a2-4358-9ed5-ecdb132bad29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:05.137075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e6996-28a2-4358-9ed5-ecdb132bad29 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:05.137554 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e6996-28a2-4358-9ed5-ecdb132bad29 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:05.138021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e6996-28a2-4358-9ed5-ecdb132bad29 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:05.139404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e6996-28a2-4358-9ed5-ecdb132bad29 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:05.139989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e6996-28a2-4358-9ed5-ecdb132bad29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:05.142719 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e6996-28a2-4358-9ed5-ecdb132bad29 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:05.178313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 275/559] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Aug 30 14:31:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:05.189796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e6996-28a2-4358-9ed5-ecdb132bad29 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:05.243503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936ee76-4ded-4ac0-94ae-fc9b901745ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:05.245899 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936ee76-4ded-4ac0-94ae-fc9b901745ad None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:05.339977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:05.351945 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e6996-28a2-4358-9ed5-ecdb132bad29 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:05.379871 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 285/560] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 14:31:05.381543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936ee76-4ded-4ac0-94ae-fc9b901745ad None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:05.400262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b371abd-74d5-423f-b86c-5cd8fedf474e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:05.400813 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b371abd-74d5-423f-b86c-5cd8fedf474e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:05.401088 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b371abd-74d5-423f-b86c-5cd8fedf474e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:05.401366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b371abd-74d5-423f-b86c-5cd8fedf474e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:05.413424 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b371abd-74d5-423f-b86c-5cd8fedf474e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:05.414929 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b371abd-74d5-423f-b86c-5cd8fedf474e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:05.418562 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:05.432690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-3b371abd-74d5-423f-b86c-5cd8fedf474e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:05.432690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:05.432690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:05.432690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:05.432690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:05.432690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:05.432690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:05.432690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:05.432690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:05.432690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:05.432690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:05.432690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:05.432690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:05.432690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:05.432690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:05.432690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:05.433726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:05.433726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:05.433726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:05.433726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:05.433726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:05.433726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:05.433726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:05.433726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:05.433726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:05.433726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:05.433726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:05.433726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:05.433726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:05.433726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:05.433726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:05.433726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:05.433726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:05.434620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:05.434620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:05.434620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:05.434620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 276/561] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:05.435478 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab60c9b-b83e-4ea9-b3c8-319d8632f51b None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:05.435478 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab60c9b-b83e-4ea9-b3c8-319d8632f51b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:05.435652 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab60c9b-b83e-4ea9-b3c8-319d8632f51b None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:05.441391 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 277/562] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:05.441391 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab60c9b-b83e-4ea9-b3c8-319d8632f51b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:05.555890 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936ee76-4ded-4ac0-94ae-fc9b901745ad None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:05.558337 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab60c9b-b83e-4ea9-b3c8-319d8632f51b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:05.603557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3373e-8f66-42cc-8ca8-71a009e1591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:05.604518 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3373e-8f66-42cc-8ca8-71a009e1591c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:05.604866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3373e-8f66-42cc-8ca8-71a009e1591c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:05.605260 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3373e-8f66-42cc-8ca8-71a009e1591c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:05.615871 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da32924-ba91-48fc-80b9-5f9671137687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:05.616907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da32924-ba91-48fc-80b9-5f9671137687 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:05.617407 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da32924-ba91-48fc-80b9-5f9671137687 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:05.617811 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da32924-ba91-48fc-80b9-5f9671137687 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:05.621433 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3373e-8f66-42cc-8ca8-71a009e1591c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:05.628142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3373e-8f66-42cc-8ca8-71a009e1591c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:05.637436 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:05.642628 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da32924-ba91-48fc-80b9-5f9671137687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:05.645030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-aaa3373e-8f66-42cc-8ca8-71a009e1591c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:05.645030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:05.645030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:05.645030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:05.645030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:05.645030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:05.645030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:05.645030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:05.645030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:05.645030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:05.645030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:05.645030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:05.645030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:05.645030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:05.645030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:05.645030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:05.646974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:05.646974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:05.646974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:05.646974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:05.646974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:05.646974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:05.646974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:05.646974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:05.646974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:05.646974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:05.646974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:05.646974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:05.646974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:05.646974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:05.646974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:05.646974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:05.646974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:05.648797 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:05.648797 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:05.648797 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:05.654654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-0da32924-ba91-48fc-80b9-5f9671137687 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:05.654654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:05.654654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:05.654654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:05.654654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:05.654654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:05.654654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:05.654654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:05.654654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:05.654654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:05.654654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:05.654654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:05.654654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:05.654654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:05.654654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:05.654654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:05.656529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:05.656529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:05.656529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:05.656529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:05.656529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:05.656529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:05.656529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:05.656529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:05.656529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:05.656529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:05.656529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:05.656529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:05.656529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:05.656529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:05.656529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:05.656529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:05.667373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:05.667373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:05.667373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:05.667373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:05.667373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:05.667373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:05.667373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:05.667373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:05.667373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:05.667373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:05.667373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:05.667373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:05.668782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 278/563] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:05.669433 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 286/564] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:31:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:05.669570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ffe33-09d1-4acf-82b6-f6f4c49e9de5 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:05.669570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ffe33-09d1-4acf-82b6-f6f4c49e9de5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:05.669570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ffe33-09d1-4acf-82b6-f6f4c49e9de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:05.669944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da32924-ba91-48fc-80b9-5f9671137687 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:05.684774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-6b8ffe33-09d1-4acf-82b6-f6f4c49e9de5 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:05.686011 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6b8ffe33-09d1-4acf-82b6-f6f4c49e9de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:05.792374 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 279/565] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:31:05.793340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ffe33-09d1-4acf-82b6-f6f4c49e9de5 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:05.834244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:05.863023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da32924-ba91-48fc-80b9-5f9671137687 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:05.928843 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ffe33-09d1-4acf-82b6-f6f4c49e9de5 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:05.939278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd72fc-e4d0-42cc-ae1a-d4d96b4100db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:05.939907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd72fc-e4d0-42cc-ae1a-d4d96b4100db None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:05.940212 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd72fc-e4d0-42cc-ae1a-d4d96b4100db None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:05.940485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd72fc-e4d0-42cc-ae1a-d4d96b4100db None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:05.941150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd72fc-e4d0-42cc-ae1a-d4d96b4100db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:05.941451 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd72fc-e4d0-42cc-ae1a-d4d96b4100db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:05.943153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd72fc-e4d0-42cc-ae1a-d4d96b4100db None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:05.960649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 287/566] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:05.961657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd72fc-e4d0-42cc-ae1a-d4d96b4100db None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:06.000739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:06.016606 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a53e75-215f-4a80-b5ce-fabbd7e0ade8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:06.018782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a53e75-215f-4a80-b5ce-fabbd7e0ade8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:06.019503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a53e75-215f-4a80-b5ce-fabbd7e0ade8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:06.020033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a53e75-215f-4a80-b5ce-fabbd7e0ade8 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:06.022461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a53e75-215f-4a80-b5ce-fabbd7e0ade8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:06.023238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a53e75-215f-4a80-b5ce-fabbd7e0ade8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:06.027437 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a53e75-215f-4a80-b5ce-fabbd7e0ade8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:06.058957 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 280/567] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:06.060885 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a53e75-215f-4a80-b5ce-fabbd7e0ade8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:06.138104 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd72fc-e4d0-42cc-ae1a-d4d96b4100db None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:06.216424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad1dc2-86b1-41ab-8855-66035a3ef297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:06.217815 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad1dc2-86b1-41ab-8855-66035a3ef297 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:06.218501 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad1dc2-86b1-41ab-8855-66035a3ef297 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:06.219149 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad1dc2-86b1-41ab-8855-66035a3ef297 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:06.221156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad1dc2-86b1-41ab-8855-66035a3ef297 None admin] RBAC: Adding query filter params (name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:06.221874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad1dc2-86b1-41ab-8855-66035a3ef297 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:06.225348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad1dc2-86b1-41ab-8855-66035a3ef297 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:06.248111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 288/568] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:31:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:06.250185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad1dc2-86b1-41ab-8855-66035a3ef297 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:06.267519 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a53e75-215f-4a80-b5ce-fabbd7e0ade8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:06.269116 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:06.323688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3cc01-b53b-4d74-81e1-3935fac7f905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:06.324744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3cc01-b53b-4d74-81e1-3935fac7f905 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:06.325181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3cc01-b53b-4d74-81e1-3935fac7f905 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:06.325604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3cc01-b53b-4d74-81e1-3935fac7f905 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:06.335960 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3cc01-b53b-4d74-81e1-3935fac7f905 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:06.337521 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3cc01-b53b-4d74-81e1-3935fac7f905 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:06.351197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-93e3cc01-b53b-4d74-81e1-3935fac7f905 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:06.351197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:06.351197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:06.351197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:06.351197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:06.351197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:06.351197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:06.351197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:06.351197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:06.351197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:06.351197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:06.351197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:06.351197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:06.351197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:06.351197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:06.352295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:06.352295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:06.352295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:06.352295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:06.352295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:06.352295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:06.352295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:06.352295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:06.352295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:06.352295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:06.352295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:06.352295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:06.352295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:06.352295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:06.352295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:06.352295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:06.352295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:06.353206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:06.353206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:06.353206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:06.353206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:06.353206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:06.353206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:06.353206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:06.353206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:06.357300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 281/569] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:06.358097 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3cc01-b53b-4d74-81e1-3935fac7f905 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:06.410839 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad1dc2-86b1-41ab-8855-66035a3ef297 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:06.423383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:06.462987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce726a-e2e8-4948-b566-9ca49f5aabcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:06.464236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce726a-e2e8-4948-b566-9ca49f5aabcc None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:06.464696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce726a-e2e8-4948-b566-9ca49f5aabcc None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:06.465158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce726a-e2e8-4948-b566-9ca49f5aabcc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:06.477758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce726a-e2e8-4948-b566-9ca49f5aabcc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:06.480411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce726a-e2e8-4948-b566-9ca49f5aabcc None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:06.483549 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3cc01-b53b-4d74-81e1-3935fac7f905 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:06.503181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-77ce726a-e2e8-4948-b566-9ca49f5aabcc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:06.503181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:06.503181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:06.503181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:06.503181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:06.503181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:06.503181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:06.503181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:06.503181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:06.503181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:06.503181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:06.503181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:06.503181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:06.503181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:06.503181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:06.504352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:06.504352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:06.504352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:06.504352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:06.504352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:06.504352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:06.504352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:06.504352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:06.504352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:06.504352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:06.504352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:06.504352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:06.504352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:06.504352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:06.504352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:06.504352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:06.504352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:06.505395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:06.505395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:06.505395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:06.505395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:06.505395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:06.505395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:06.505395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:06.505395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:06.505848 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 289/570] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:06.507129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce726a-e2e8-4948-b566-9ca49f5aabcc None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:06.527585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316d7a9-f05b-40cc-bd9f-63abcfc66cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:06.528534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316d7a9-f05b-40cc-bd9f-63abcfc66cca None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:06.529052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316d7a9-f05b-40cc-bd9f-63abcfc66cca None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:06.529588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316d7a9-f05b-40cc-bd9f-63abcfc66cca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:06.542183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316d7a9-f05b-40cc-bd9f-63abcfc66cca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:06.544189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316d7a9-f05b-40cc-bd9f-63abcfc66cca None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:06.564052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-7316d7a9-f05b-40cc-bd9f-63abcfc66cca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:06.564052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:06.564052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:06.564052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:06.564052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:06.564052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:06.564052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:06.564052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:06.564052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:06.564052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:06.564052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:06.564052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:06.564052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:06.564052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:06.564052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:06.564052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:06.565936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:06.565936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:06.565936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:06.565936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:06.565936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:06.565936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:06.565936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:06.565936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:06.565936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:06.565936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:06.565936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:06.565936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:06.565936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:06.565936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:06.565936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:06.565936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:06.565936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:06.567702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:06.567702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:06.567702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:06.567702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 282/571] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:06.567702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316d7a9-f05b-40cc-bd9f-63abcfc66cca None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:06.651053 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:06.712441 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce726a-e2e8-4948-b566-9ca49f5aabcc None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:06.732304 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316d7a9-f05b-40cc-bd9f-63abcfc66cca None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:06.734289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:06.762156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafc1e6-f906-4265-834a-7d0b41bbeb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:06.763149 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafc1e6-f906-4265-834a-7d0b41bbeb0e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:06.763582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafc1e6-f906-4265-834a-7d0b41bbeb0e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:06.764029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafc1e6-f906-4265-834a-7d0b41bbeb0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:06.781286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fafc1e6-f906-4265-834a-7d0b41bbeb0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:06.790976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-5fafc1e6-f906-4265-834a-7d0b41bbeb0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:06.790976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:06.790976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:06.790976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:06.790976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:06.790976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:06.790976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:06.790976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:06.790976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:06.790976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:06.790976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:06.790976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:06.790976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:06.790976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:06.790976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:06.790976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:06.792234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:06.792234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:06.792234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:06.792234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:06.792234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:06.792234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:06.792234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:06.792234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:06.792234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:06.792234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:06.792234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:06.792234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:06.792234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:06.792234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:06.792234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:06.792234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:06.793199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:06.793199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:06.793199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:06.793199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:06.793199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:06.793199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:06.793199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:06.793199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:06.793199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:06.793199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:06.793199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:06.793199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:06.793842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 290/572] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:31:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:06.794324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafc1e6-f906-4265-834a-7d0b41bbeb0e None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:06.798454 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ee356-59c8-49c4-aca7-7ab9a8bb45e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:06.799277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ee356-59c8-49c4-aca7-7ab9a8bb45e9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:06.799715 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ee356-59c8-49c4-aca7-7ab9a8bb45e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:06.800181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ee356-59c8-49c4-aca7-7ab9a8bb45e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:06.801143 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ee356-59c8-49c4-aca7-7ab9a8bb45e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:06.801487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ee356-59c8-49c4-aca7-7ab9a8bb45e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:06.803782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ee356-59c8-49c4-aca7-7ab9a8bb45e9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:06.822817 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 283/573] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:06.824240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ee356-59c8-49c4-aca7-7ab9a8bb45e9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:06.942110 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafc1e6-f906-4265-834a-7d0b41bbeb0e None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:06.963831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ee356-59c8-49c4-aca7-7ab9a8bb45e9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:06.984599 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:06.997192 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec8b09-8ce6-4916-84e1-f20a7d7718d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:06.998165 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec8b09-8ce6-4916-84e1-f20a7d7718d5 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:06.998517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec8b09-8ce6-4916-84e1-f20a7d7718d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:06.999037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec8b09-8ce6-4916-84e1-f20a7d7718d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:07.000294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec8b09-8ce6-4916-84e1-f20a7d7718d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:07.000695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec8b09-8ce6-4916-84e1-f20a7d7718d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:07.003196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec8b09-8ce6-4916-84e1-f20a7d7718d5 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:07.016887 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401107e0-94aa-4b4c-b4f5-60816b8e421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:07.017594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401107e0-94aa-4b4c-b4f5-60816b8e421e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:07.017937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401107e0-94aa-4b4c-b4f5-60816b8e421e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:07.018320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401107e0-94aa-4b4c-b4f5-60816b8e421e None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:07.019115 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401107e0-94aa-4b4c-b4f5-60816b8e421e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:07.019501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401107e0-94aa-4b4c-b4f5-60816b8e421e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:07.021087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401107e0-94aa-4b4c-b4f5-60816b8e421e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:07.022302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 291/574] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:07.023653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab842c31-416c-4c33-ad1a-30aba411088b None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:07.023941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab842c31-416c-4c33-ad1a-30aba411088b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:07.024254 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab842c31-416c-4c33-ad1a-30aba411088b None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:07.025626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 292/575] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:07.026418 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab842c31-416c-4c33-ad1a-30aba411088b None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:07.043970 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 284/576] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:07.045344 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401107e0-94aa-4b4c-b4f5-60816b8e421e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:07.085966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:07.171323 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab842c31-416c-4c33-ad1a-30aba411088b None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:07.197130 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401107e0-94aa-4b4c-b4f5-60816b8e421e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:07.231585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3fff6-8135-4314-9cbb-df3c8342ff6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:07.232669 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3fff6-8135-4314-9cbb-df3c8342ff6f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:07.233150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3fff6-8135-4314-9cbb-df3c8342ff6f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:07.233629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3fff6-8135-4314-9cbb-df3c8342ff6f None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:07.234494 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3fff6-8135-4314-9cbb-df3c8342ff6f None admin] RBAC: Adding query filter params (name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:07.234882 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e9db8-7d46-4322-9b7c-557fe4ffab14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:07.235025 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3fff6-8135-4314-9cbb-df3c8342ff6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:07.235510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e9db8-7d46-4322-9b7c-557fe4ffab14 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:07.235784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e9db8-7d46-4322-9b7c-557fe4ffab14 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:07.236089 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e9db8-7d46-4322-9b7c-557fe4ffab14 None admin] PATH_INFO: `/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/cc95bbdafb0c4f34ad0662bc2e3ba4ec/roles/f0fd911d7c9246aeb3d67299473a9f42` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:07.237592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3fff6-8135-4314-9cbb-df3c8342ff6f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:07.252656 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e9db8-7d46-4322-9b7c-557fe4ffab14 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9c57a581324217ad2e64a0c9c7fdee, user_id=cc95bbdafb0c4f34ad0662bc2e3ba4ec, role_id=f0fd911d7c9246aeb3d67299473a9f42)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:07.253331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 293/577] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:31:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:07.254435 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e9db8-7d46-4322-9b7c-557fe4ffab14 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:07.255608 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f376d-ca11-4ee4-9a20-6e7001c98afc None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:07.256124 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f376d-ca11-4ee4-9a20-6e7001c98afc None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:07.256615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f376d-ca11-4ee4-9a20-6e7001c98afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:07.271824 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 285/578] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:31:07 2022] PUT /identity/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/cc95bbdafb0c4f34ad0662bc2e3ba4ec/roles/f0fd911d7c9246aeb3d67299473a9f42 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:31:07.272747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e9db8-7d46-4322-9b7c-557fe4ffab14 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:07.290620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a62f376d-ca11-4ee4-9a20-6e7001c98afc None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:07.294081 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a62f376d-ca11-4ee4-9a20-6e7001c98afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:07.374595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:07.408118 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 294/579] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:31:07.408118 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f376d-ca11-4ee4-9a20-6e7001c98afc None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:07.420240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:07.465064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e9db8-7d46-4322-9b7c-557fe4ffab14 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:07.516443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c074795-98ef-4cb6-b661-e4b57378aeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:07.517008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c074795-98ef-4cb6-b661-e4b57378aeed None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:07.517287 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c074795-98ef-4cb6-b661-e4b57378aeed None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:07.517555 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c074795-98ef-4cb6-b661-e4b57378aeed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:07.518326 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c074795-98ef-4cb6-b661-e4b57378aeed None admin] RBAC: Adding query filter params (role.id=f0fd911d7c9246aeb3d67299473a9f42, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=d5d98f5adfaf410580481417e7ba1638) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:07.518635 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c074795-98ef-4cb6-b661-e4b57378aeed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:07.521481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c074795-98ef-4cb6-b661-e4b57378aeed None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:07.528468 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 286/580] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:07 2022] GET /identity/v3/role_assignments?user.id=d5d98f5adfaf410580481417e7ba1638&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=f0fd911d7c9246aeb3d67299473a9f42 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:07.530210 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c074795-98ef-4cb6-b661-e4b57378aeed None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:07.568899 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f376d-ca11-4ee4-9a20-6e7001c98afc None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:07.611598 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d49f9-48c2-417b-9f09-50a363a13b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:07.612826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d49f9-48c2-417b-9f09-50a363a13b9b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:07.613120 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d49f9-48c2-417b-9f09-50a363a13b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:07.613400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d49f9-48c2-417b-9f09-50a363a13b9b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:07.622211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d49f9-48c2-417b-9f09-50a363a13b9b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:07.624664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d49f9-48c2-417b-9f09-50a363a13b9b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:07.642672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-f64d49f9-48c2-417b-9f09-50a363a13b9b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:31:07.642672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:07.642672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:07.642672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:07.642672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:07.642672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:07.642672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:07.642672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:07.642672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:07.642672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:07.642672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:07.642672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:07.642672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:07.642672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:07.642672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:07.642672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:07.645167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:07.647312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:07.647312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:07.647312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:07.647312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:07.647312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:07.647312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:07.647312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:07.647312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:07.647312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:31:07.647312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:07.647312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 295/581] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:31:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:07.652402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d49f9-48c2-417b-9f09-50a363a13b9b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:07.705143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:07.776121 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d49f9-48c2-417b-9f09-50a363a13b9b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:07.811864 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c074795-98ef-4cb6-b661-e4b57378aeed None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:07.817138 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f570e02-e0a3-4011-929f-9dcdc011b738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:07.818064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f570e02-e0a3-4011-929f-9dcdc011b738 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:07.818502 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f570e02-e0a3-4011-929f-9dcdc011b738 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:07.818897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f570e02-e0a3-4011-929f-9dcdc011b738 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:07.819781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f570e02-e0a3-4011-929f-9dcdc011b738 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:07.820245 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f570e02-e0a3-4011-929f-9dcdc011b738 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:07.821844 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f570e02-e0a3-4011-929f-9dcdc011b738 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:07.849528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 296/582] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Aug 30 14:31:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:31:07.855478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f570e02-e0a3-4011-929f-9dcdc011b738 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:07.904541 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9031b7b-53f0-4022-b4fc-fe1dbf7e6d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:07.907109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9031b7b-53f0-4022-b4fc-fe1dbf7e6d0e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:07.907568 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9031b7b-53f0-4022-b4fc-fe1dbf7e6d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:07.908031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9031b7b-53f0-4022-b4fc-fe1dbf7e6d0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:07.928546 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:07.947319 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9031b7b-53f0-4022-b4fc-fe1dbf7e6d0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:07.950209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9031b7b-53f0-4022-b4fc-fe1dbf7e6d0e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:07.983527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-e9031b7b-53f0-4022-b4fc-fe1dbf7e6d0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:07.983527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:07.983527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:07.983527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:07.983527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:07.983527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:07.983527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:07.983527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:07.983527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:07.983527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:07.983527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:07.983527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:07.983527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:07.983527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:07.983527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:07.983527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:07.984603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:07.984603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:07.984603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:07.984603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:07.984603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:07.984603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:07.984603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:07.984603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:07.984603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:07.984603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:07.984603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:07.984603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:07.984603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:07.984603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:07.984603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:07.984603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:07.984603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:07.985512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:07.985512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:07.985512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:07.987844 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 287/583] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 459 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:07.993675 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9031b7b-53f0-4022-b4fc-fe1dbf7e6d0e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:08.008855 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:08.016241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f570e02-e0a3-4011-929f-9dcdc011b738 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:08.088021 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c78f2a-0bb6-4239-b6b3-ae12df7ce78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:08.089053 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c78f2a-0bb6-4239-b6b3-ae12df7ce78e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:08.089535 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c78f2a-0bb6-4239-b6b3-ae12df7ce78e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:08.090009 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c78f2a-0bb6-4239-b6b3-ae12df7ce78e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:08.109774 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c78f2a-0bb6-4239-b6b3-ae12df7ce78e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:08.112707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c78f2a-0bb6-4239-b6b3-ae12df7ce78e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:08.129769 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9031b7b-53f0-4022-b4fc-fe1dbf7e6d0e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:08.141477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-68c78f2a-0bb6-4239-b6b3-ae12df7ce78e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:08.141477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:08.141477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:08.141477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:08.141477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:08.141477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:08.141477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:08.141477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:08.141477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:08.141477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:08.141477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:08.141477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:08.141477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:08.141477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:08.141477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:08.141477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:08.142521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:08.142521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:08.142521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:08.142521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:08.142521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:08.142521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:08.142521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:08.142521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:08.142521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:08.142521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:08.142521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:08.142521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:08.142521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:08.142521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:08.142521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:08.142521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:08.142521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:08.143417 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:08.143417 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:08.143417 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:08.144036 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 297/584] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:31:08.145331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c78f2a-0bb6-4239-b6b3-ae12df7ce78e None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:08.182075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8831df-a1ba-4916-86a5-501fc1809773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:08.183187 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8831df-a1ba-4916-86a5-501fc1809773 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:08.183672 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8831df-a1ba-4916-86a5-501fc1809773 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:08.184219 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8831df-a1ba-4916-86a5-501fc1809773 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:08.199540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8831df-a1ba-4916-86a5-501fc1809773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:08.207714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-4e8831df-a1ba-4916-86a5-501fc1809773 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:08.207714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:08.207714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:08.207714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:08.207714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:08.207714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:08.207714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:08.207714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:08.207714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:08.207714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:08.207714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:08.207714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:08.207714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:08.207714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:08.207714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:08.207714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:08.208785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:08.208785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:08.208785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:08.208785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:08.208785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:08.208785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:08.208785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:08.208785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:08.208785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:08.208785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:08.208785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:08.208785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:08.208785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:08.208785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:08.208785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:08.208785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:08.209612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:08.209612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:08.209612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:08.209612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:08.209612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:08.209612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:08.209612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:08.209612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:08.209612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:08.209612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:08.209612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:08.209612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:08.212255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 288/585] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:31:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:08.213367 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8831df-a1ba-4916-86a5-501fc1809773 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:08.292607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:08.340857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c78f2a-0bb6-4239-b6b3-ae12df7ce78e None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:08.354287 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8831df-a1ba-4916-86a5-501fc1809773 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:08.385772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be8e2a-99a5-4ef2-bd15-ee31216b1226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:08.386759 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be8e2a-99a5-4ef2-bd15-ee31216b1226 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:08.387190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be8e2a-99a5-4ef2-bd15-ee31216b1226 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:08.387834 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be8e2a-99a5-4ef2-bd15-ee31216b1226 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:08.388714 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be8e2a-99a5-4ef2-bd15-ee31216b1226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:08.389019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be8e2a-99a5-4ef2-bd15-ee31216b1226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:08.391109 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be8e2a-99a5-4ef2-bd15-ee31216b1226 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:08.412100 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 298/586] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:08.413375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be8e2a-99a5-4ef2-bd15-ee31216b1226 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:08.426061 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f63df-8aa8-4a4c-b7ac-1a2128f32e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:08.427237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f63df-8aa8-4a4c-b7ac-1a2128f32e52 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:08.427702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f63df-8aa8-4a4c-b7ac-1a2128f32e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:08.428545 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:08.429567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f63df-8aa8-4a4c-b7ac-1a2128f32e52 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:08.430964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f63df-8aa8-4a4c-b7ac-1a2128f32e52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:08.431652 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f63df-8aa8-4a4c-b7ac-1a2128f32e52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:08.434663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f63df-8aa8-4a4c-b7ac-1a2128f32e52 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:08.461923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 289/587] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:08.463895 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f63df-8aa8-4a4c-b7ac-1a2128f32e52 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:08.542150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be8e2a-99a5-4ef2-bd15-ee31216b1226 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:08.615094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e2909-f88e-487c-b368-685d4a92d001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:08.617566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e2909-f88e-487c-b368-685d4a92d001 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:08.618449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e2909-f88e-487c-b368-685d4a92d001 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:08.619219 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e2909-f88e-487c-b368-685d4a92d001 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:08.620935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e2909-f88e-487c-b368-685d4a92d001 None admin] RBAC: Adding query filter params (name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:08.621817 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e2909-f88e-487c-b368-685d4a92d001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:08.622415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f63df-8aa8-4a4c-b7ac-1a2128f32e52 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:08.625350 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e2909-f88e-487c-b368-685d4a92d001 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:08.644622 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:08.652497 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 299/588] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:31:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:08.654383 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e2909-f88e-487c-b368-685d4a92d001 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:08.667230 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467207b0-cd85-4c2a-9cfd-de0bf961e92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:08.668107 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467207b0-cd85-4c2a-9cfd-de0bf961e92f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:08.668488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467207b0-cd85-4c2a-9cfd-de0bf961e92f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:08.668858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467207b0-cd85-4c2a-9cfd-de0bf961e92f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:08.678229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467207b0-cd85-4c2a-9cfd-de0bf961e92f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:08.679894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467207b0-cd85-4c2a-9cfd-de0bf961e92f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:08.702516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-467207b0-cd85-4c2a-9cfd-de0bf961e92f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:31:08.702516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:08.702516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:08.702516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:08.702516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:08.702516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:08.702516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:08.702516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:08.702516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:08.702516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:08.702516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:08.702516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:08.702516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:08.702516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:08.702516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:08.702516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:08.704688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:08.706516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:08.706516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:08.706516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:08.706516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:08.706516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:08.706516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:08.706516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:08.706516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:08.706516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:31:08.706516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:08.711099 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 290/589] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:31:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:08.713025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467207b0-cd85-4c2a-9cfd-de0bf961e92f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:08.794222 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:08.847716 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467207b0-cd85-4c2a-9cfd-de0bf961e92f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:08.855345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e2909-f88e-487c-b368-685d4a92d001 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:08.906395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77755201-5400-4da7-809d-9b6ea9eb6a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:08.906970 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77755201-5400-4da7-809d-9b6ea9eb6a5a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:08.907249 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77755201-5400-4da7-809d-9b6ea9eb6a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:08.907523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77755201-5400-4da7-809d-9b6ea9eb6a5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:08.924825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77755201-5400-4da7-809d-9b6ea9eb6a5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:08.927483 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77755201-5400-4da7-809d-9b6ea9eb6a5a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:08.939985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe491b-9384-4c04-9818-edf7840aa7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:08.941693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe491b-9384-4c04-9818-edf7840aa7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:08.942619 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe491b-9384-4c04-9818-edf7840aa7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:08.943377 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe491b-9384-4c04-9818-edf7840aa7e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:08.957215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-77755201-5400-4da7-809d-9b6ea9eb6a5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:08.957215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:08.957215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:08.957215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:08.957215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:08.957215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:08.957215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:08.957215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:08.957215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:08.957215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:08.957215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:08.957215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:08.957215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:08.957215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:08.957215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:08.957215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:08.959313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:08.959313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:08.959313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:08.959313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:08.959313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:08.959313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:08.959313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:08.959313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:08.959313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:08.959313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:08.959313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:08.959313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:08.959313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:08.959313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:08.959313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:08.959313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:08.959313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:08.960862 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:08.960862 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:08.960862 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:08.960862 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 291/590] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:08.960862 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77755201-5400-4da7-809d-9b6ea9eb6a5a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:08.968870 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe491b-9384-4c04-9818-edf7840aa7e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:08.972440 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe491b-9384-4c04-9818-edf7840aa7e9 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:08.983883 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:08.998153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-89fe491b-9384-4c04-9818-edf7840aa7e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:08.998153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:08.998153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:08.998153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:08.998153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:08.998153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:08.998153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:08.998153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:08.998153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:08.998153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:08.998153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:08.998153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:08.998153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:08.998153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:08.998153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:09.000087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:09.000087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:09.000087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:09.000087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:09.000087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:09.000087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:09.000087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:09.000087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:09.000087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:09.000087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:09.000087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:09.000087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:09.000087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:09.000087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:09.000087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:09.000087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:09.000087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:09.001809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:09.001809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:09.001809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:09.001809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:09.001809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:09.001809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:09.001809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:09.001809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:09.006157 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 300/591] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:09.007988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe491b-9384-4c04-9818-edf7840aa7e9 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:09.125815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77755201-5400-4da7-809d-9b6ea9eb6a5a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:09.179884 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf7f0a-fcdc-4b8e-80b9-14651dff2198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:09.180517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf7f0a-fcdc-4b8e-80b9-14651dff2198 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:09.180825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf7f0a-fcdc-4b8e-80b9-14651dff2198 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:09.181072 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf7f0a-fcdc-4b8e-80b9-14651dff2198 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:09.182106 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf7f0a-fcdc-4b8e-80b9-14651dff2198 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:09.182288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf7f0a-fcdc-4b8e-80b9-14651dff2198 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:09.183698 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf7f0a-fcdc-4b8e-80b9-14651dff2198 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:09.204671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe491b-9384-4c04-9818-edf7840aa7e9 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:09.219029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 292/592] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Aug 30 14:31:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:09.220607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf7f0a-fcdc-4b8e-80b9-14651dff2198 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:09.266856 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:09.276915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ef9bc-0070-4515-b6b4-9ea02838a72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:09.278643 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ef9bc-0070-4515-b6b4-9ea02838a72c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:09.279259 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ef9bc-0070-4515-b6b4-9ea02838a72c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:09.279854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ef9bc-0070-4515-b6b4-9ea02838a72c None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:09.281495 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ef9bc-0070-4515-b6b4-9ea02838a72c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:09.282135 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ef9bc-0070-4515-b6b4-9ea02838a72c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:09.285166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ef9bc-0070-4515-b6b4-9ea02838a72c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:09.316040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 301/593] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:09.316040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ef9bc-0070-4515-b6b4-9ea02838a72c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:09.390097 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:09.427680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ef9bc-0070-4515-b6b4-9ea02838a72c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:09.432481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf7f0a-fcdc-4b8e-80b9-14651dff2198 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:09.470733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de08524-2c4d-4e6a-8549-08d0dc1d93ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:09.471421 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de08524-2c4d-4e6a-8549-08d0dc1d93ed None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:09.471698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de08524-2c4d-4e6a-8549-08d0dc1d93ed None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:09.471967 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de08524-2c4d-4e6a-8549-08d0dc1d93ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:09.484560 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de08524-2c4d-4e6a-8549-08d0dc1d93ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:09.486066 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de08524-2c4d-4e6a-8549-08d0dc1d93ed None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:09.504034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-1de08524-2c4d-4e6a-8549-08d0dc1d93ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:09.504034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:09.504034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:09.504034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:09.504034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:09.504034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:09.504034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:09.504034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:09.504034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:09.504034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:09.504034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:09.504034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:09.504034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:09.504034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:09.504034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:09.504034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:09.505858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:09.505858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:09.505858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:09.505858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:09.505858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:09.505858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:09.505858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:09.505858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:09.505858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:09.505858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:09.505858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:09.505858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:09.505858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:09.505858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:09.505858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:09.505858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:09.505858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:09.507622 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:09.507622 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:09.507622 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:09.507622 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 302/594] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:09.507622 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de08524-2c4d-4e6a-8549-08d0dc1d93ed None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:09.518432 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbadb7f-e91c-4aab-9db7-9a93bbc2610b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:09.520568 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbadb7f-e91c-4aab-9db7-9a93bbc2610b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:09.521885 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbadb7f-e91c-4aab-9db7-9a93bbc2610b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:09.523033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbadb7f-e91c-4aab-9db7-9a93bbc2610b None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:09.525767 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbadb7f-e91c-4aab-9db7-9a93bbc2610b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:09.527135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbadb7f-e91c-4aab-9db7-9a93bbc2610b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:09.531173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbadb7f-e91c-4aab-9db7-9a93bbc2610b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:09.568210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:09.568487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 293/595] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:09.569810 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbadb7f-e91c-4aab-9db7-9a93bbc2610b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:09.672707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de08524-2c4d-4e6a-8549-08d0dc1d93ed None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:09.722390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87bc678-0077-409b-aab4-6d84a93d4c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:09.723335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87bc678-0077-409b-aab4-6d84a93d4c5e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:09.723803 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87bc678-0077-409b-aab4-6d84a93d4c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:09.724325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87bc678-0077-409b-aab4-6d84a93d4c5e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:09.729276 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbadb7f-e91c-4aab-9db7-9a93bbc2610b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:09.739648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87bc678-0077-409b-aab4-6d84a93d4c5e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:09.741600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87bc678-0077-409b-aab4-6d84a93d4c5e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:09.754965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-f87bc678-0077-409b-aab4-6d84a93d4c5e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:31:09.754965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:09.754965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:09.754965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:09.754965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:09.754965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:09.754965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:09.754965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:09.754965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:09.754965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:09.754965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:09.754965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:09.754965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:09.754965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:09.754965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:09.754965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:09.756029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:09.757158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:09.757158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:09.757158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:09.757158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:09.757158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:09.757158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:09.757158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:09.757158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:09.757158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:31:09.757158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:09.760895 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 303/596] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:31:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:09.762169 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fa375-724d-4e3c-9e70-eb5e305adcc4 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:09.762828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fa375-724d-4e3c-9e70-eb5e305adcc4 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:09.762828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fa375-724d-4e3c-9e70-eb5e305adcc4 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:09.768276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 304/597] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:09.768276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fa375-724d-4e3c-9e70-eb5e305adcc4 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:09.784247 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb8b24-229d-4156-a60e-374b9a568662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:09.785336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb8b24-229d-4156-a60e-374b9a568662 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:09.785828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb8b24-229d-4156-a60e-374b9a568662 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:09.786316 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb8b24-229d-4156-a60e-374b9a568662 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:09.787485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb8b24-229d-4156-a60e-374b9a568662 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:09.788051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb8b24-229d-4156-a60e-374b9a568662 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:09.790668 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb8b24-229d-4156-a60e-374b9a568662 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:09.803428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:09.819935 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 294/598] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:09.821224 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb8b24-229d-4156-a60e-374b9a568662 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:09.914667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fa375-724d-4e3c-9e70-eb5e305adcc4 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:09.919493 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:09.970806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb8b24-229d-4156-a60e-374b9a568662 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:09.979980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a360f8-f7d6-4940-b117-fdfdc169ddba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:09.981040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a360f8-f7d6-4940-b117-fdfdc169ddba None admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:09.981578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a360f8-f7d6-4940-b117-fdfdc169ddba None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:09.982090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a360f8-f7d6-4940-b117-fdfdc169ddba None admin] PATH_INFO: `/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/2788fece401a4a7ebdb516afe2dd8ec8/roles/f0fd911d7c9246aeb3d67299473a9f42` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:10.001737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a360f8-f7d6-4940-b117-fdfdc169ddba None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9c57a581324217ad2e64a0c9c7fdee, user_id=2788fece401a4a7ebdb516afe2dd8ec8, role_id=f0fd911d7c9246aeb3d67299473a9f42)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:10.003964 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a360f8-f7d6-4940-b117-fdfdc169ddba None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:10.022318 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 305/599] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:31:09 2022] PUT /identity/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/2788fece401a4a7ebdb516afe2dd8ec8/roles/f0fd911d7c9246aeb3d67299473a9f42 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:31:10.024510 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bfcc2-0f56-4a40-a074-1b9a3e41194c None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:10.025007 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bfcc2-0f56-4a40-a074-1b9a3e41194c None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:10.025406 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bfcc2-0f56-4a40-a074-1b9a3e41194c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:10.038887 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734df3f0-3d45-4c8a-827e-46d91c82e75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:10.039942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734df3f0-3d45-4c8a-827e-46d91c82e75c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:10.040545 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734df3f0-3d45-4c8a-827e-46d91c82e75c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:10.041258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734df3f0-3d45-4c8a-827e-46d91c82e75c None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:10.042295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734df3f0-3d45-4c8a-827e-46d91c82e75c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:10.042908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734df3f0-3d45-4c8a-827e-46d91c82e75c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:10.045451 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734df3f0-3d45-4c8a-827e-46d91c82e75c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:10.056853 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-c66bfcc2-0f56-4a40-a074-1b9a3e41194c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:10.059127 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c66bfcc2-0f56-4a40-a074-1b9a3e41194c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:10.079008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 295/600] 10.222.0.25 () {56 vars in 1139 bytes} [Tue Aug 30 14:31:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:10.081896 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734df3f0-3d45-4c8a-827e-46d91c82e75c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:10.186626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:10.189375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 306/601] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:31:10.191058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bfcc2-0f56-4a40-a074-1b9a3e41194c None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:10.258923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734df3f0-3d45-4c8a-827e-46d91c82e75c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:10.322949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b730cb7-e6cd-4d36-82c2-4283928e79f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:10.324237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b730cb7-e6cd-4d36-82c2-4283928e79f3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:10.324871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b730cb7-e6cd-4d36-82c2-4283928e79f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:10.325365 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b730cb7-e6cd-4d36-82c2-4283928e79f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:10.328419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:10.339249 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b730cb7-e6cd-4d36-82c2-4283928e79f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:10.342065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b730cb7-e6cd-4d36-82c2-4283928e79f3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:10.349151 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bfcc2-0f56-4a40-a074-1b9a3e41194c None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:10.360375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-6b730cb7-e6cd-4d36-82c2-4283928e79f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:10.360375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:10.360375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:10.360375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:10.360375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:10.360375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:10.360375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:10.360375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:10.360375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:10.360375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:10.360375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:10.360375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:10.360375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:10.360375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:10.360375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:10.361332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:10.361332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:10.361332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:10.361332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:10.361332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:10.361332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:10.361332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:10.361332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:10.361332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:10.361332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:10.361332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:10.361332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:10.361332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:10.361332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:10.361332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:10.361332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:10.361332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:10.362209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:10.362209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:10.362209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:10.362209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:10.362209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:10.362209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:10.362209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:10.362209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:10.362635 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 296/602] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:10.370662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b730cb7-e6cd-4d36-82c2-4283928e79f3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:10.400268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fe9a2-46f2-41a2-9fae-d7a1b5be125d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:10.400930 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fe9a2-46f2-41a2-9fae-d7a1b5be125d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:10.401206 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fe9a2-46f2-41a2-9fae-d7a1b5be125d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:10.401473 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fe9a2-46f2-41a2-9fae-d7a1b5be125d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:10.424697 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fe9a2-46f2-41a2-9fae-d7a1b5be125d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:10.426185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fe9a2-46f2-41a2-9fae-d7a1b5be125d None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:10.454390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-b32fe9a2-46f2-41a2-9fae-d7a1b5be125d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:10.454390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:10.454390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:10.454390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:10.454390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:10.454390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:10.454390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:10.454390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:10.454390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:10.454390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:10.454390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:10.454390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:10.454390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:10.454390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:10.454390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:10.454390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:10.456550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:10.456550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:10.456550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:10.456550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:10.456550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:10.456550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:10.456550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:10.456550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:10.456550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:10.456550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:10.456550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:10.456550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:10.456550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:10.456550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:10.456550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:10.456550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:10.456550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:10.458934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:10.458934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:10.458934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:10.458934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 307/603] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:10.465709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fe9a2-46f2-41a2-9fae-d7a1b5be125d None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:10.603670 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b730cb7-e6cd-4d36-82c2-4283928e79f3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:10.636790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:10.650605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933b0d3-d8dc-444d-9d00-ba6dbda84873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:10.651912 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933b0d3-d8dc-444d-9d00-ba6dbda84873 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:10.652227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933b0d3-d8dc-444d-9d00-ba6dbda84873 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:10.652501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933b0d3-d8dc-444d-9d00-ba6dbda84873 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:10.653734 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933b0d3-d8dc-444d-9d00-ba6dbda84873 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:10.654044 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933b0d3-d8dc-444d-9d00-ba6dbda84873 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:10.656173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933b0d3-d8dc-444d-9d00-ba6dbda84873 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:10.680297 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 297/604] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:10.681584 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933b0d3-d8dc-444d-9d00-ba6dbda84873 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:10.695572 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fe9a2-46f2-41a2-9fae-d7a1b5be125d None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:10.757973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4004b-1144-4f7f-aa03-c9a161cc9175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:10.758733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4004b-1144-4f7f-aa03-c9a161cc9175 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:10.759136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4004b-1144-4f7f-aa03-c9a161cc9175 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:10.759459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4004b-1144-4f7f-aa03-c9a161cc9175 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:10.760537 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4004b-1144-4f7f-aa03-c9a161cc9175 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:10.760985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4004b-1144-4f7f-aa03-c9a161cc9175 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:10.763215 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4004b-1144-4f7f-aa03-c9a161cc9175 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:10.767585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:10.789788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 308/605] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:10.789788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4004b-1144-4f7f-aa03-c9a161cc9175 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:10.871822 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933b0d3-d8dc-444d-9d00-ba6dbda84873 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:10.929053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f4c99-8bb1-4bc7-8643-c28bac425470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:10.930004 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f4c99-8bb1-4bc7-8643-c28bac425470 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:10.930455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f4c99-8bb1-4bc7-8643-c28bac425470 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:10.930886 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f4c99-8bb1-4bc7-8643-c28bac425470 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:10.955162 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4004b-1144-4f7f-aa03-c9a161cc9175 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:10.959689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f4c99-8bb1-4bc7-8643-c28bac425470 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:10.967748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-420f4c99-8bb1-4bc7-8643-c28bac425470 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:10.967748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:10.967748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:10.967748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:10.967748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:10.967748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:10.967748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:10.967748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:10.967748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:10.967748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:10.967748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:10.967748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:10.967748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:10.967748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:10.967748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:10.967748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:10.969504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:10.969504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:10.969504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:10.969504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:10.969504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:10.969504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:10.969504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:10.969504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:10.969504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:10.969504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:10.969504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:10.969504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:10.969504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:10.969504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:10.969504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:10.969504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:10.973395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:10.973395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:10.973395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:10.973395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:10.973395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:10.973395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:10.973395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:10.973395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:10.973395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:10.973395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:10.973395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:10.973395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:10.973395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 298/606] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:31:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:10.973395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfffff5-ee64-4e3b-8c96-3fd057df960c None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:10.975690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfffff5-ee64-4e3b-8c96-3fd057df960c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:10.975690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfffff5-ee64-4e3b-8c96-3fd057df960c None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:10.977027 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 299/607] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:10.978296 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfffff5-ee64-4e3b-8c96-3fd057df960c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:11.028235 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:11.034717 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95d320-9fc3-4220-b5a3-f162094d40ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:11.036184 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95d320-9fc3-4220-b5a3-f162094d40ae None admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:11.036919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95d320-9fc3-4220-b5a3-f162094d40ae None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:11.037629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95d320-9fc3-4220-b5a3-f162094d40ae None admin] PATH_INFO: `/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/69de9368993f402cb6325e5022e064d4/roles/f0fd911d7c9246aeb3d67299473a9f42` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:11.065552 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95d320-9fc3-4220-b5a3-f162094d40ae None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9c57a581324217ad2e64a0c9c7fdee, user_id=69de9368993f402cb6325e5022e064d4, role_id=f0fd911d7c9246aeb3d67299473a9f42)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:11.067010 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95d320-9fc3-4220-b5a3-f162094d40ae None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:11.091279 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 309/608] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:31:10 2022] PUT /identity/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/69de9368993f402cb6325e5022e064d4/roles/f0fd911d7c9246aeb3d67299473a9f42 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:31:11.093499 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95d320-9fc3-4220-b5a3-f162094d40ae None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:11.150926 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfffff5-ee64-4e3b-8c96-3fd057df960c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:11.160713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:11.200990 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964dccee-2941-4dca-90b5-8fe24505a532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:11.201797 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dccee-2941-4dca-90b5-8fe24505a532 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:11.202138 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dccee-2941-4dca-90b5-8fe24505a532 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:11.203889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dccee-2941-4dca-90b5-8fe24505a532 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:11.220195 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964dccee-2941-4dca-90b5-8fe24505a532 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:11.223498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964dccee-2941-4dca-90b5-8fe24505a532 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:11.248911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-964dccee-2941-4dca-90b5-8fe24505a532 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:11.248911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:11.248911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:11.248911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:11.248911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:11.248911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:11.248911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:11.248911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:11.248911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:11.248911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:11.248911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:11.248911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:11.248911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:11.248911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:11.248911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:11.251121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:11.251121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:11.251121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:11.251121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:11.251121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:11.251121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:11.251121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:11.251121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:11.251121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:11.251121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:11.251121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:11.251121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:11.251121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:11.251121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:11.251121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:11.251121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:11.251121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:11.253339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:11.253339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:11.253339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:11.253339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:11.253339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:11.253339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:11.253339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:11.253339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:11.253339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 300/609] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:11.254468 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adc37d-8b0b-4fd2-bc05-f15bf349d634 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:11.255718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adc37d-8b0b-4fd2-bc05-f15bf349d634 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:11.255718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adc37d-8b0b-4fd2-bc05-f15bf349d634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:11.271813 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95d320-9fc3-4220-b5a3-f162094d40ae None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:11.291813 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-19adc37d-8b0b-4fd2-bc05-f15bf349d634 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:11.294561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-19adc37d-8b0b-4fd2-bc05-f15bf349d634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:11.336695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1bdc3-b2cd-40bf-b805-4d819c3a8ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:11.337770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1bdc3-b2cd-40bf-b805-4d819c3a8ddd None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:11.338340 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1bdc3-b2cd-40bf-b805-4d819c3a8ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:11.338816 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1bdc3-b2cd-40bf-b805-4d819c3a8ddd None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:11.339735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1bdc3-b2cd-40bf-b805-4d819c3a8ddd None admin] RBAC: Adding query filter params (name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:11.340345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1bdc3-b2cd-40bf-b805-4d819c3a8ddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:11.342013 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1bdc3-b2cd-40bf-b805-4d819c3a8ddd None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:11.356286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 310/610] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:31:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:11.356286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1bdc3-b2cd-40bf-b805-4d819c3a8ddd None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:11.438010 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 301/611] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:31:11.439815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19adc37d-8b0b-4fd2-bc05-f15bf349d634 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:11.500652 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:11.548350 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1bdc3-b2cd-40bf-b805-4d819c3a8ddd None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:11.570449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:11.627924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfecbb-5170-4ba1-b0f0-68ed905adef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:11.627924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfecbb-5170-4ba1-b0f0-68ed905adef3 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:11.627924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfecbb-5170-4ba1-b0f0-68ed905adef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:11.627924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfecbb-5170-4ba1-b0f0-68ed905adef3 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:11.631826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfecbb-5170-4ba1-b0f0-68ed905adef3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:11.631826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfecbb-5170-4ba1-b0f0-68ed905adef3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:11.634278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfecbb-5170-4ba1-b0f0-68ed905adef3 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:11.651122 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19adc37d-8b0b-4fd2-bc05-f15bf349d634 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:11.658046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 311/612] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:11.663858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfecbb-5170-4ba1-b0f0-68ed905adef3 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:11.724755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580c747-5f45-4346-a026-63ea9beafbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:11.725453 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580c747-5f45-4346-a026-63ea9beafbb5 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:11.725795 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580c747-5f45-4346-a026-63ea9beafbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:11.726134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580c747-5f45-4346-a026-63ea9beafbb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:11.746727 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580c747-5f45-4346-a026-63ea9beafbb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:11.752613 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580c747-5f45-4346-a026-63ea9beafbb5 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:11.773513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-1580c747-5f45-4346-a026-63ea9beafbb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:11.773513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:11.773513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:11.773513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:11.773513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:11.773513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:11.773513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:11.773513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:11.773513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:11.773513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:11.773513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:11.773513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:11.773513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:11.773513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:11.773513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:11.773513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:11.775300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:11.775300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:11.775300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:11.775300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:11.775300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:11.775300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:11.775300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:11.775300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:11.775300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:11.775300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:11.775300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:11.775300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:11.775300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:11.775300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:11.775300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:11.775300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:11.775300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:11.777072 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:11.777072 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:11.777072 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:11.779835 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 302/613] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:11.781552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580c747-5f45-4346-a026-63ea9beafbb5 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:11.890487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:11.918225 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580c747-5f45-4346-a026-63ea9beafbb5 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:11.959101 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e620f-e941-46f0-8915-13b5c0ab1dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:11.959710 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfecbb-5170-4ba1-b0f0-68ed905adef3 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:11.960432 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e620f-e941-46f0-8915-13b5c0ab1dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:11.960432 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e620f-e941-46f0-8915-13b5c0ab1dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:11.960743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e620f-e941-46f0-8915-13b5c0ab1dc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:11.961517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e620f-e941-46f0-8915-13b5c0ab1dc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:11.961915 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e620f-e941-46f0-8915-13b5c0ab1dc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:11.963426 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e620f-e941-46f0-8915-13b5c0ab1dc0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:11.982428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 303/614] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:11.983574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e620f-e941-46f0-8915-13b5c0ab1dc0 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:12.028569 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a29050-f4d3-4d55-8e82-4f488941b6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:12.029802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a29050-f4d3-4d55-8e82-4f488941b6ff None admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:12.030585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a29050-f4d3-4d55-8e82-4f488941b6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:12.031158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a29050-f4d3-4d55-8e82-4f488941b6ff None admin] PATH_INFO: `/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/932fce741c5a4deba79a310a14e6bd2c/roles/f0fd911d7c9246aeb3d67299473a9f42` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:12.059246 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a29050-f4d3-4d55-8e82-4f488941b6ff None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9c57a581324217ad2e64a0c9c7fdee, user_id=932fce741c5a4deba79a310a14e6bd2c, role_id=f0fd911d7c9246aeb3d67299473a9f42)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:12.062765 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a29050-f4d3-4d55-8e82-4f488941b6ff None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:12.089955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 312/615] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:31:11 2022] PUT /identity/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/932fce741c5a4deba79a310a14e6bd2c/roles/f0fd911d7c9246aeb3d67299473a9f42 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:31:12.091110 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a29050-f4d3-4d55-8e82-4f488941b6ff None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:12.112912 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e620f-e941-46f0-8915-13b5c0ab1dc0 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:12.160581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:12.191691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309071c9-aa3e-4b0f-ac37-ffa268188757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:12.192509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309071c9-aa3e-4b0f-ac37-ffa268188757 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:12.192859 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309071c9-aa3e-4b0f-ac37-ffa268188757 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:12.193142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309071c9-aa3e-4b0f-ac37-ffa268188757 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:12.213974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:12.219694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309071c9-aa3e-4b0f-ac37-ffa268188757 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:12.229499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-309071c9-aa3e-4b0f-ac37-ffa268188757 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:12.229499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:12.229499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:12.229499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:12.229499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:12.229499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:12.229499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:12.229499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:12.229499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:12.229499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:12.229499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:12.229499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:12.229499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:12.229499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:12.229499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:12.229499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:12.230561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:12.230561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:12.230561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:12.230561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:12.230561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:12.230561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:12.230561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:12.230561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:12.230561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:12.230561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:12.230561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:12.230561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:12.230561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:12.230561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:12.230561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:12.230561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:12.231390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:12.231390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:12.231390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:12.231390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:12.231390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:12.231390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:12.231390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:12.231390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:12.231390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:12.231390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:12.231390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:12.231390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:12.231390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 304/616] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:31:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:12.244812 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a29050-f4d3-4d55-8e82-4f488941b6ff None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:12.245170 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309071c9-aa3e-4b0f-ac37-ffa268188757 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:12.291026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71601c-768b-45af-b549-7dfcdf28acc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:12.292153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71601c-768b-45af-b549-7dfcdf28acc5 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:12.292562 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71601c-768b-45af-b549-7dfcdf28acc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:12.292927 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71601c-768b-45af-b549-7dfcdf28acc5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:12.301429 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71601c-768b-45af-b549-7dfcdf28acc5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:12.303084 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71601c-768b-45af-b549-7dfcdf28acc5 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:12.320638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-2a71601c-768b-45af-b549-7dfcdf28acc5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:31:12.320638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:12.320638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:12.320638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:12.320638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:12.320638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:12.320638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:12.320638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:12.320638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:12.320638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:12.320638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:12.320638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:12.320638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:12.320638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:12.320638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:12.320638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:12.321683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:12.322874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:12.322874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:12.322874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:12.322874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:12.322874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:12.322874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:12.322874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:12.322874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:12.322874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:31:12.322874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:12.322874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 313/617] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Aug 30 14:31:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:12.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028304f-0405-4c40-b559-19c88280e0a2 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:12.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028304f-0405-4c40-b559-19c88280e0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:12.325635 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028304f-0405-4c40-b559-19c88280e0a2 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:12.327454 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 314/618] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:12.331038 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028304f-0405-4c40-b559-19c88280e0a2 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:12.441619 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309071c9-aa3e-4b0f-ac37-ffa268188757 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:12.481750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:12.516573 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028304f-0405-4c40-b559-19c88280e0a2 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:12.524677 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f25659-9281-4f03-8a83-8799d4b1ba8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:12.525299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f25659-9281-4f03-8a83-8799d4b1ba8e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:12.525577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f25659-9281-4f03-8a83-8799d4b1ba8e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:12.525844 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f25659-9281-4f03-8a83-8799d4b1ba8e None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:12.526536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f25659-9281-4f03-8a83-8799d4b1ba8e None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:12.526833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f25659-9281-4f03-8a83-8799d4b1ba8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:12.528491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f25659-9281-4f03-8a83-8799d4b1ba8e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:12.539088 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 305/619] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:31:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:12.540570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c83cf-858e-496d-8663-c5ac010052d5 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:12.540863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c83cf-858e-496d-8663-c5ac010052d5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:12.541135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c83cf-858e-496d-8663-c5ac010052d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:12.564415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-521c83cf-858e-496d-8663-c5ac010052d5 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:12.565733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-521c83cf-858e-496d-8663-c5ac010052d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:12.598387 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc8137-72b9-46c4-a689-2a766cc3c8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:12.600244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc8137-72b9-46c4-a689-2a766cc3c8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:12.601228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc8137-72b9-46c4-a689-2a766cc3c8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:12.602528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc8137-72b9-46c4-a689-2a766cc3c8e6 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:12.603885 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc8137-72b9-46c4-a689-2a766cc3c8e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:12.604975 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc8137-72b9-46c4-a689-2a766cc3c8e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:12.608040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc8137-72b9-46c4-a689-2a766cc3c8e6 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:12.642781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 315/620] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Aug 30 14:31:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:12.652033 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc8137-72b9-46c4-a689-2a766cc3c8e6 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:12.661583 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:12.684027 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 306/621] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:31:12.684027 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c83cf-858e-496d-8663-c5ac010052d5 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:12.875969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc8137-72b9-46c4-a689-2a766cc3c8e6 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:12.879845 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c83cf-858e-496d-8663-c5ac010052d5 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:12.956378 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd693d07-25c6-430a-ab3c-741d8b4bafdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:12.959938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd693d07-25c6-430a-ab3c-741d8b4bafdd None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:12.962311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd693d07-25c6-430a-ab3c-741d8b4bafdd None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:12.964529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd693d07-25c6-430a-ab3c-741d8b4bafdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:12.984385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:13.003695 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469d687-2292-4424-b358-9f973bbdfd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:13.004983 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469d687-2292-4424-b358-9f973bbdfd75 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:13.006758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd693d07-25c6-430a-ab3c-741d8b4bafdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:13.007400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469d687-2292-4424-b358-9f973bbdfd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:13.008037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469d687-2292-4424-b358-9f973bbdfd75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:13.010985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd693d07-25c6-430a-ab3c-741d8b4bafdd None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:13.040925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-bd693d07-25c6-430a-ab3c-741d8b4bafdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:13.040925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:13.040925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:13.040925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:13.040925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:13.040925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:13.040925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:13.040925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:13.040925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:13.040925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:13.040925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:13.040925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:13.040925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:13.040925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:13.040925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:13.040925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:13.042094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:13.042094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:13.042094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:13.042094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:13.042094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:13.042094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:13.042094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:13.042094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:13.042094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:13.042094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:13.042094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:13.042094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:13.042094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:13.042094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:13.042094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:13.042094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:13.042094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:13.043142 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:13.043142 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:13.043142 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:13.043142 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 316/622] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:13.043539 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd693d07-25c6-430a-ab3c-741d8b4bafdd None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:13.132575 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:13.146532 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469d687-2292-4424-b358-9f973bbdfd75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:13.148612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469d687-2292-4424-b358-9f973bbdfd75 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:13.263242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd693d07-25c6-430a-ab3c-741d8b4bafdd None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:13.294818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 307/623] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:31:13.296921 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469d687-2292-4424-b358-9f973bbdfd75 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:13.320133 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041192ad-98da-4c22-94d5-b9a177b90be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:13.320789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041192ad-98da-4c22-94d5-b9a177b90be6 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:13.321119 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041192ad-98da-4c22-94d5-b9a177b90be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:13.321399 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041192ad-98da-4c22-94d5-b9a177b90be6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:13.334478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041192ad-98da-4c22-94d5-b9a177b90be6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:13.340047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-041192ad-98da-4c22-94d5-b9a177b90be6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:13.340047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:13.340047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:13.340047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:13.340047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:13.340047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:13.340047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:13.340047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:13.340047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:13.340047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:13.340047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:13.340047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:13.340047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:13.340047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:13.340047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:13.340047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:13.341881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:13.341881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:13.341881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:13.341881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:13.341881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:13.341881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:13.341881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:13.341881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:13.341881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:13.341881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:13.341881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:13.341881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:13.341881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:13.341881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:13.341881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:13.341881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:13.343620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:13.343620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:13.343620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:13.343620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:13.343620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:13.343620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:13.343620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:13.343620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:13.343620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:13.343620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:13.343620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:13.343620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:13.343620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 317/624] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:31:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:13.343620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041192ad-98da-4c22-94d5-b9a177b90be6 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:13.436474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:13.460948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:13.474452 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469d687-2292-4424-b358-9f973bbdfd75 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:13.524496 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041192ad-98da-4c22-94d5-b9a177b90be6 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:13.546017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfce7b-4427-4c88-b3ec-47cfbce2b922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:13.547949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfce7b-4427-4c88-b3ec-47cfbce2b922 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:13.548721 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfce7b-4427-4c88-b3ec-47cfbce2b922 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:13.549538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfce7b-4427-4c88-b3ec-47cfbce2b922 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:13.550842 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfce7b-4427-4c88-b3ec-47cfbce2b922 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:13.551787 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfce7b-4427-4c88-b3ec-47cfbce2b922 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:13.554780 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfce7b-4427-4c88-b3ec-47cfbce2b922 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:13.576640 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 308/625] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:13.578579 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfce7b-4427-4c88-b3ec-47cfbce2b922 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:13.592270 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103848f-1dcb-4c32-99bd-085248cd0a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:13.593405 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103848f-1dcb-4c32-99bd-085248cd0a34 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:13.593852 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103848f-1dcb-4c32-99bd-085248cd0a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:13.594314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103848f-1dcb-4c32-99bd-085248cd0a34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:13.619591 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103848f-1dcb-4c32-99bd-085248cd0a34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:13.622345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103848f-1dcb-4c32-99bd-085248cd0a34 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:13.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-5103848f-1dcb-4c32-99bd-085248cd0a34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:13.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:13.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:13.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:13.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:13.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:13.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:13.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:13.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:13.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:13.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:13.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:13.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:13.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:13.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:13.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:13.653936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:13.653936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:13.653936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:13.653936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:13.653936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:13.653936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:13.653936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:13.653936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:13.653936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:13.653936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:13.653936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:13.653936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:13.653936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:13.653936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:13.653936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:13.653936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:13.653936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:13.655599 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:13.655599 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:13.655599 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:13.661212 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 318/626] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:13.662256 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103848f-1dcb-4c32-99bd-085248cd0a34 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:13.776377 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:13.780260 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103848f-1dcb-4c32-99bd-085248cd0a34 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:13.786357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfce7b-4427-4c88-b3ec-47cfbce2b922 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:13.821918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72528125-3f7c-4e66-aa1b-e9432a98cdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:13.822583 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72528125-3f7c-4e66-aa1b-e9432a98cdba None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:13.822862 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72528125-3f7c-4e66-aa1b-e9432a98cdba None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:13.823133 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72528125-3f7c-4e66-aa1b-e9432a98cdba None admin] PATH_INFO: `/v3/users/swift` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:13.832930 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72528125-3f7c-4e66-aa1b-e9432a98cdba None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:13.834530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72528125-3f7c-4e66-aa1b-e9432a98cdba None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:13.847971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-72528125-3f7c-4e66-aa1b-e9432a98cdba None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:31:13.847971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:13.847971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:13.847971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:13.847971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:13.847971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:13.847971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:13.847971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:13.847971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:13.847971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:13.847971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:13.847971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:13.847971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:13.847971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:13.847971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:13.847971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:13.849795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:13.851665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:13.851665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:13.851665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:13.851665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:13.851665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:13.851665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:13.851665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:13.851665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:13.851665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:31:13.851665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:13.851665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 319/627] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:13.851665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72528125-3f7c-4e66-aa1b-e9432a98cdba None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:13.857609 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd5622-c5d5-4c7e-af20-bec8024881ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:13.859077 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd5622-c5d5-4c7e-af20-bec8024881ed None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:13.859810 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd5622-c5d5-4c7e-af20-bec8024881ed None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:13.860523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd5622-c5d5-4c7e-af20-bec8024881ed None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:13.861918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd5622-c5d5-4c7e-af20-bec8024881ed None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:13.862697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd5622-c5d5-4c7e-af20-bec8024881ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:13.865368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd5622-c5d5-4c7e-af20-bec8024881ed None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:13.880430 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 309/628] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:31:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:13.882311 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd5622-c5d5-4c7e-af20-bec8024881ed None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:13.906924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:14.014264 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72528125-3f7c-4e66-aa1b-e9432a98cdba None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:14.015091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd5622-c5d5-4c7e-af20-bec8024881ed None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:14.057197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e12ed-3927-4276-8bc9-ba529bace90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:14.057733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e12ed-3927-4276-8bc9-ba529bace90a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:14.057984 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e12ed-3927-4276-8bc9-ba529bace90a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:14.058300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e12ed-3927-4276-8bc9-ba529bace90a None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:14.058987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e12ed-3927-4276-8bc9-ba529bace90a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:14.059288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e12ed-3927-4276-8bc9-ba529bace90a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:14.059814 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a006e73-17cd-4a3d-ad55-5781c3d92fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:14.060603 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a006e73-17cd-4a3d-ad55-5781c3d92fca None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:14.060868 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e12ed-3927-4276-8bc9-ba529bace90a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:14.061112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a006e73-17cd-4a3d-ad55-5781c3d92fca None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:14.061642 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a006e73-17cd-4a3d-ad55-5781c3d92fca None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:14.062466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a006e73-17cd-4a3d-ad55-5781c3d92fca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:14.062998 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a006e73-17cd-4a3d-ad55-5781c3d92fca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:14.065210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a006e73-17cd-4a3d-ad55-5781c3d92fca None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:14.080938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 320/629] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:14.086372 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 310/630] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Aug 30 14:31:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:14.087179 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747b1b5-7a4d-4f77-b731-2fc60f845dd9 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:14.087337 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e12ed-3927-4276-8bc9-ba529bace90a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:14.088059 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747b1b5-7a4d-4f77-b731-2fc60f845dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:14.088480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747b1b5-7a4d-4f77-b731-2fc60f845dd9 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:14.091913 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 321/631] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:14.094492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747b1b5-7a4d-4f77-b731-2fc60f845dd9 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:14.126368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:14.181860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:14.222932 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747b1b5-7a4d-4f77-b731-2fc60f845dd9 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:14.237157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e12ed-3927-4276-8bc9-ba529bace90a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:14.259800 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab95ed7-bc69-47d4-9e27-f1ea7f041c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:14.260483 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab95ed7-bc69-47d4-9e27-f1ea7f041c57 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:14.260754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab95ed7-bc69-47d4-9e27-f1ea7f041c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:14.261129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab95ed7-bc69-47d4-9e27-f1ea7f041c57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:14.273493 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab95ed7-bc69-47d4-9e27-f1ea7f041c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:14.275089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab95ed7-bc69-47d4-9e27-f1ea7f041c57 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:14.292668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-6ab95ed7-bc69-47d4-9e27-f1ea7f041c57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:14.292668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:14.292668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:14.292668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:14.292668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:14.292668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:14.292668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:14.292668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:14.292668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:14.292668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:14.292668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:14.292668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:14.292668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:14.292668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:14.292668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:14.292668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:14.293704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:14.293704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:14.293704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:14.293704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:14.293704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:14.293704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:14.293704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:14.293704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:14.293704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:14.293704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:14.293704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:14.293704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:14.293704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:14.293704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:14.293704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:14.293704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:14.293704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:14.294594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:14.294594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:14.294594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:14.298029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff5132-2932-4ed0-be82-1d9debf97b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:14.298803 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 322/632] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:14.299486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff5132-2932-4ed0-be82-1d9debf97b31 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:14.299891 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab95ed7-bc69-47d4-9e27-f1ea7f041c57 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:14.300181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff5132-2932-4ed0-be82-1d9debf97b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:14.300548 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff5132-2932-4ed0-be82-1d9debf97b31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:14.319081 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff5132-2932-4ed0-be82-1d9debf97b31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:14.321788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff5132-2932-4ed0-be82-1d9debf97b31 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:14.341016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-4aff5132-2932-4ed0-be82-1d9debf97b31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:14.341016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:14.341016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:14.341016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:14.341016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:14.341016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:14.341016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:14.341016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:14.341016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:14.341016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:14.341016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:14.341016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:14.341016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:14.341016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:14.341016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:14.341016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:14.342039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:14.342039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:14.342039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:14.342039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:14.342039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:14.342039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:14.342039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:14.342039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:14.342039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:14.342039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:14.342039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:14.342039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:14.342039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:14.342039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:14.342039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:14.342039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:14.342039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:14.343788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:14.343788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:14.343788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:14.345466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 311/633] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:14.346948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906d0ea-0358-4dfd-9b38-bf96a73a358c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:14.347461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906d0ea-0358-4dfd-9b38-bf96a73a358c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:14.347861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906d0ea-0358-4dfd-9b38-bf96a73a358c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:14.368081 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a906d0ea-0358-4dfd-9b38-bf96a73a358c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:14.369313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a906d0ea-0358-4dfd-9b38-bf96a73a358c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:14.428033 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:14.461399 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 312/634] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:31:14.463291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906d0ea-0358-4dfd-9b38-bf96a73a358c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:14.467317 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab95ed7-bc69-47d4-9e27-f1ea7f041c57 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:14.523321 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8e748-3460-493e-93d4-191a8f9abbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:14.524454 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8e748-3460-493e-93d4-191a8f9abbb7 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:14.524940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8e748-3460-493e-93d4-191a8f9abbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:14.525429 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8e748-3460-493e-93d4-191a8f9abbb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:14.533543 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:14.539111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8e748-3460-493e-93d4-191a8f9abbb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:14.541950 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8e748-3460-493e-93d4-191a8f9abbb7 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:14.564240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-71c8e748-3460-493e-93d4-191a8f9abbb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:14.564240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:14.564240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:14.564240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:14.564240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:14.564240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:14.564240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:14.564240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:14.564240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:14.564240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:14.564240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:14.564240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:14.564240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:14.564240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:14.564240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:14.574161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:14.574161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:14.574161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:14.574161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:14.574161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:14.574161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:14.574161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:14.574161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:14.574161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:14.574161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:14.574161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:14.574161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:14.574161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:14.574161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:14.574161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:14.574161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:14.574161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:14.576274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:14.576274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:14.576274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:14.576274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:14.576274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:14.576274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:14.576274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:14.576274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:14.576274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 323/635] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:14.576274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8e748-3460-493e-93d4-191a8f9abbb7 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:14.629531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906d0ea-0358-4dfd-9b38-bf96a73a358c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:14.687872 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f0948-6440-493b-a013-f42bf71632d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:14.688710 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f0948-6440-493b-a013-f42bf71632d6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:14.689134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f0948-6440-493b-a013-f42bf71632d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:14.689531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f0948-6440-493b-a013-f42bf71632d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:14.690373 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f0948-6440-493b-a013-f42bf71632d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:14.690806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f0948-6440-493b-a013-f42bf71632d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:14.692454 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f0948-6440-493b-a013-f42bf71632d6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:14.710734 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 313/636] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:14.712556 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1c771-32a6-4c4a-8e60-8bc34d3f6ca6 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:14.712890 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1c771-32a6-4c4a-8e60-8bc34d3f6ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:14.713218 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1c771-32a6-4c4a-8e60-8bc34d3f6ca6 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:14.720089 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 314/637] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:14.720089 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1c771-32a6-4c4a-8e60-8bc34d3f6ca6 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:14.752114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8e748-3460-493e-93d4-191a8f9abbb7 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:14.787695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:14.794462 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1c126-dec4-4656-8bfb-e4b59a11c5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:14.795172 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1c126-dec4-4656-8bfb-e4b59a11c5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:14.795493 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1c126-dec4-4656-8bfb-e4b59a11c5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:14.796040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1c126-dec4-4656-8bfb-e4b59a11c5e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:14.797474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1c126-dec4-4656-8bfb-e4b59a11c5e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:14.797990 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1c126-dec4-4656-8bfb-e4b59a11c5e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:14.800555 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1c126-dec4-4656-8bfb-e4b59a11c5e7 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:14.824058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 324/638] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:14.824967 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1c126-dec4-4656-8bfb-e4b59a11c5e7 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:14.845744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1c771-32a6-4c4a-8e60-8bc34d3f6ca6 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:14.863254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:14.894093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5d1e6-1454-46f3-9610-9aaeda25d50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:14.895152 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5d1e6-1454-46f3-9610-9aaeda25d50d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:14.895597 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5d1e6-1454-46f3-9610-9aaeda25d50d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:14.896061 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5d1e6-1454-46f3-9610-9aaeda25d50d None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:14.897107 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5d1e6-1454-46f3-9610-9aaeda25d50d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:14.897613 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5d1e6-1454-46f3-9610-9aaeda25d50d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:14.900081 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5d1e6-1454-46f3-9610-9aaeda25d50d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:14.917538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 315/639] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:14.919761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ad9c6-dd7b-4434-ba1a-00f994fdb24a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:14.920269 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ad9c6-dd7b-4434-ba1a-00f994fdb24a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:14.920871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ad9c6-dd7b-4434-ba1a-00f994fdb24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:14.945007 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-105ad9c6-dd7b-4434-ba1a-00f994fdb24a None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:14.946839 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-105ad9c6-dd7b-4434-ba1a-00f994fdb24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:14.956119 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1c126-dec4-4656-8bfb-e4b59a11c5e7 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:15.002187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06826f-df19-45d5-8774-a4b26b768a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:15.002967 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06826f-df19-45d5-8774-a4b26b768a1b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:15.003396 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06826f-df19-45d5-8774-a4b26b768a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:15.003770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06826f-df19-45d5-8774-a4b26b768a1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:15.011956 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06826f-df19-45d5-8774-a4b26b768a1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:15.013643 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06826f-df19-45d5-8774-a4b26b768a1b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:15.027083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-ca06826f-df19-45d5-8774-a4b26b768a1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:15.027083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:15.027083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:15.027083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:15.027083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:15.027083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:15.027083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:15.027083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:15.027083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:15.027083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:15.027083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:15.027083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:15.027083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:15.027083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:15.027083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:15.029703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:15.029703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:15.029703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:15.029703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:15.029703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:15.029703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:15.029703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:15.029703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:15.029703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:15.029703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:15.029703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:15.029703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:15.029703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:15.029703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:15.029703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:15.029703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:15.029703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:15.031400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:15.031400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:15.031400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:15.031400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:15.031400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:15.031400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:15.031400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:15.031400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:15.031400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 325/640] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:15.031400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06826f-df19-45d5-8774-a4b26b768a1b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:15.047209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 316/641] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:31:15.048534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ad9c6-dd7b-4434-ba1a-00f994fdb24a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:15.128765 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:15.207874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ad9c6-dd7b-4434-ba1a-00f994fdb24a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:15.209789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:15.218716 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06826f-df19-45d5-8774-a4b26b768a1b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:15.269343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eaa035-db6c-41e8-8d0b-83abb040b9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:15.270418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eaa035-db6c-41e8-8d0b-83abb040b9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:15.271261 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eaa035-db6c-41e8-8d0b-83abb040b9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:15.271766 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eaa035-db6c-41e8-8d0b-83abb040b9a6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:15.285536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eaa035-db6c-41e8-8d0b-83abb040b9a6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:15.287111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a532de-7a81-4964-a8f6-857b52e3e601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:15.288212 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a532de-7a81-4964-a8f6-857b52e3e601 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:15.288318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eaa035-db6c-41e8-8d0b-83abb040b9a6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:15.288657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a532de-7a81-4964-a8f6-857b52e3e601 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:15.289376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a532de-7a81-4964-a8f6-857b52e3e601 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:15.308933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a532de-7a81-4964-a8f6-857b52e3e601 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:15.310409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-70eaa035-db6c-41e8-8d0b-83abb040b9a6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:31:15.310409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:15.310409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:15.310409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:15.310409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:15.310409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:15.310409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:15.310409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:15.310409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:15.310409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:15.310409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:15.310409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:15.310409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:15.310409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:15.310409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:15.310409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:15.311520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:15.312689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:15.312689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:15.312689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:15.312689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:15.312689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:15.312689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:15.312689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:15.312689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:15.312689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:31:15.312689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:15.319024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-28a532de-7a81-4964-a8f6-857b52e3e601 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:15.319024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:15.319024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:15.319024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:15.319024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:15.319024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:15.319024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:15.319024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:15.319024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:15.319024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:15.319024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:15.319024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:15.319024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:15.319024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:15.319024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:15.319024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:15.321190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:15.321190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:15.321190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:15.321190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:15.321190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:15.321190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:15.321190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:15.321190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:15.321190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:15.321190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:15.321190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:15.321190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:15.321190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:15.321190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:15.321190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:15.321190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:15.323299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:15.323299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:15.323299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:15.323299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:15.323299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:15.323299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:15.323299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:15.323299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:15.323299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:15.323299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:15.323299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:15.323299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:15.324861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 317/642] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:31:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:15.324861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eaa035-db6c-41e8-8d0b-83abb040b9a6 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:15.328285 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 326/643] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:31:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:15.329680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a532de-7a81-4964-a8f6-857b52e3e601 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:15.458224 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eaa035-db6c-41e8-8d0b-83abb040b9a6 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:15.493117 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81506d16-e03d-4e6f-84d6-765fac981180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:15.493751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81506d16-e03d-4e6f-84d6-765fac981180 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:15.494021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81506d16-e03d-4e6f-84d6-765fac981180 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:15.494290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81506d16-e03d-4e6f-84d6-765fac981180 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:15.495030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81506d16-e03d-4e6f-84d6-765fac981180 None admin] RBAC: Adding query filter params (role.id=f0fd911d7c9246aeb3d67299473a9f42, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=cc95bbdafb0c4f34ad0662bc2e3ba4ec) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:15.495335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81506d16-e03d-4e6f-84d6-765fac981180 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:15.496021 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a532de-7a81-4964-a8f6-857b52e3e601 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:15.496830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81506d16-e03d-4e6f-84d6-765fac981180 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:15.504103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 318/644] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:15 2022] GET /identity/v3/role_assignments?user.id=cc95bbdafb0c4f34ad0662bc2e3ba4ec&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=f0fd911d7c9246aeb3d67299473a9f42 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:15.504985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81506d16-e03d-4e6f-84d6-765fac981180 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:15.526715 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:15.555685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9598cd-2b3e-47da-beb2-30a5237c025c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:15.556347 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9598cd-2b3e-47da-beb2-30a5237c025c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:15.556627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9598cd-2b3e-47da-beb2-30a5237c025c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:15.556892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9598cd-2b3e-47da-beb2-30a5237c025c None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:15.557616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9598cd-2b3e-47da-beb2-30a5237c025c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:15.557921 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9598cd-2b3e-47da-beb2-30a5237c025c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:15.559417 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9598cd-2b3e-47da-beb2-30a5237c025c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:15.560740 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:15.580041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 327/645] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:15.580041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9598cd-2b3e-47da-beb2-30a5237c025c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:15.713696 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81506d16-e03d-4e6f-84d6-765fac981180 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:15.741460 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9598cd-2b3e-47da-beb2-30a5237c025c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:15.771420 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21ffac-01a1-41bd-bd93-276d77defd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:15.772085 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21ffac-01a1-41bd-bd93-276d77defd77 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:15.772370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21ffac-01a1-41bd-bd93-276d77defd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:15.772633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21ffac-01a1-41bd-bd93-276d77defd77 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:15.773394 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21ffac-01a1-41bd-bd93-276d77defd77 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:15.773696 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21ffac-01a1-41bd-bd93-276d77defd77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:15.775163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21ffac-01a1-41bd-bd93-276d77defd77 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:15.786778 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822694e4-0c0a-414f-a538-0ac169a2ff2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:15.787410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822694e4-0c0a-414f-a538-0ac169a2ff2e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:15.787683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822694e4-0c0a-414f-a538-0ac169a2ff2e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:15.787953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822694e4-0c0a-414f-a538-0ac169a2ff2e None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:15.788684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822694e4-0c0a-414f-a538-0ac169a2ff2e None admin] RBAC: Adding query filter params (name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:15.789021 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822694e4-0c0a-414f-a538-0ac169a2ff2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:15.790432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822694e4-0c0a-414f-a538-0ac169a2ff2e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:15.797914 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 319/646] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Aug 30 14:31:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:15.801618 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21ffac-01a1-41bd-bd93-276d77defd77 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:15.804169 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 328/647] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:31:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:15.805089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822694e4-0c0a-414f-a538-0ac169a2ff2e None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:15.849595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:15.941471 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822694e4-0c0a-414f-a538-0ac169a2ff2e None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:15.942755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:15.966221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21ffac-01a1-41bd-bd93-276d77defd77 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:15.993892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6bcc6-5af4-49df-bef7-3e8d879d779f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:15.994566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6bcc6-5af4-49df-bef7-3e8d879d779f None admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:15.994840 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6bcc6-5af4-49df-bef7-3e8d879d779f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:15.995110 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6bcc6-5af4-49df-bef7-3e8d879d779f None admin] PATH_INFO: `/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/d5d98f5adfaf410580481417e7ba1638/roles/f0fd911d7c9246aeb3d67299473a9f42` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:16.012658 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6bcc6-5af4-49df-bef7-3e8d879d779f None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9c57a581324217ad2e64a0c9c7fdee, user_id=d5d98f5adfaf410580481417e7ba1638, role_id=f0fd911d7c9246aeb3d67299473a9f42)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:16.014185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6bcc6-5af4-49df-bef7-3e8d879d779f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:16.018038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeada13-650b-4ffc-8f64-424b788a10ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:16.018856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeada13-650b-4ffc-8f64-424b788a10ee None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:16.019139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeada13-650b-4ffc-8f64-424b788a10ee None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:16.019458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeada13-650b-4ffc-8f64-424b788a10ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:16.033549 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeada13-650b-4ffc-8f64-424b788a10ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:16.040353 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 329/648] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:31:15 2022] PUT /identity/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/d5d98f5adfaf410580481417e7ba1638/roles/f0fd911d7c9246aeb3d67299473a9f42 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:31:16.040353 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6bcc6-5af4-49df-bef7-3e8d879d779f None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:16.041000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-faeada13-650b-4ffc-8f64-424b788a10ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:16.041000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:16.041000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:16.041000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:16.041000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:16.041000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:16.041000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:16.041000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:16.041000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:16.041000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:16.041000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:16.041000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:16.041000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:16.041000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:16.041000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:16.041000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:16.042136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:16.042136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:16.042136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:16.042136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:16.042136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:16.042136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:16.042136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:16.042136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:16.042136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:16.042136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:16.042136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:16.042136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:16.042136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:16.042136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:16.042136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:16.042136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:16.043168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:16.043168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:16.043168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:16.043168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:16.043168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:16.043168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:16.043168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:16.043168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:16.043168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:16.043168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:16.043168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:16.043168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:16.043168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 320/649] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:31:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:16.043168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeada13-650b-4ffc-8f64-424b788a10ee None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:16.162237 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:16.184302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6bcc6-5af4-49df-bef7-3e8d879d779f None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:16.193365 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeada13-650b-4ffc-8f64-424b788a10ee None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:16.227323 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c21667-f71f-490d-894f-7787f41d672a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:16.227982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c21667-f71f-490d-894f-7787f41d672a None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:16.228392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c21667-f71f-490d-894f-7787f41d672a None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:16.228758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c21667-f71f-490d-894f-7787f41d672a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:16.259602 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216f01f-24d8-4ba3-b0cc-8cea9f359219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:16.260795 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216f01f-24d8-4ba3-b0cc-8cea9f359219 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:16.261240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216f01f-24d8-4ba3-b0cc-8cea9f359219 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:16.261517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216f01f-24d8-4ba3-b0cc-8cea9f359219 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:16.282794 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216f01f-24d8-4ba3-b0cc-8cea9f359219 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:16.286513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216f01f-24d8-4ba3-b0cc-8cea9f359219 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:16.291565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:16.312056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-c216f01f-24d8-4ba3-b0cc-8cea9f359219 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:16.312056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:16.312056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:16.312056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:16.312056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:16.312056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:16.312056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:16.312056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:16.312056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:16.312056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:16.312056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:16.312056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:16.312056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:16.312056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:16.312056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:16.312056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:16.314467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:16.314467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:16.314467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:16.314467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:16.314467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:16.314467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:16.314467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:16.314467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:16.314467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:16.314467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:16.314467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:16.314467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:16.314467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:16.314467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:16.314467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:16.314467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:16.314467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:16.316410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:16.316410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:16.316410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:16.316410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 321/650] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:16.316410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216f01f-24d8-4ba3-b0cc-8cea9f359219 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:16.331060 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c21667-f71f-490d-894f-7787f41d672a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:16.333283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c21667-f71f-490d-894f-7787f41d672a None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:16.460652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:16.472359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 330/651] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:31:16.474253 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c21667-f71f-490d-894f-7787f41d672a None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:16.496123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216f01f-24d8-4ba3-b0cc-8cea9f359219 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:16.555140 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094d03f-fcf3-4c1b-933d-52d08d350291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:16.555941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094d03f-fcf3-4c1b-933d-52d08d350291 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:16.556347 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094d03f-fcf3-4c1b-933d-52d08d350291 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:16.556709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094d03f-fcf3-4c1b-933d-52d08d350291 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:16.557767 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094d03f-fcf3-4c1b-933d-52d08d350291 None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:16.558137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094d03f-fcf3-4c1b-933d-52d08d350291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:16.559982 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094d03f-fcf3-4c1b-933d-52d08d350291 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:16.572528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 322/652] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:31:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:16.573919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094d03f-fcf3-4c1b-933d-52d08d350291 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:16.655477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c21667-f71f-490d-894f-7787f41d672a None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:16.686452 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:16.732617 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d6e60-0fe7-4a25-9930-fa9841727bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:16.734054 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d6e60-0fe7-4a25-9930-fa9841727bcc None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:16.734846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d6e60-0fe7-4a25-9930-fa9841727bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:16.735419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d6e60-0fe7-4a25-9930-fa9841727bcc None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:16.736905 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d6e60-0fe7-4a25-9930-fa9841727bcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:16.737533 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d6e60-0fe7-4a25-9930-fa9841727bcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:16.740528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d6e60-0fe7-4a25-9930-fa9841727bcc None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:16.772165 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 331/653] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:16.772165 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d6e60-0fe7-4a25-9930-fa9841727bcc None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:16.787793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094d03f-fcf3-4c1b-933d-52d08d350291 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:16.855337 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b63e76-81fb-4105-9771-6f11080f65dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:16.856427 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b63e76-81fb-4105-9771-6f11080f65dc None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:16.856964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b63e76-81fb-4105-9771-6f11080f65dc None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:16.857436 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b63e76-81fb-4105-9771-6f11080f65dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:16.867045 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:16.883433 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b63e76-81fb-4105-9771-6f11080f65dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:16.886650 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b63e76-81fb-4105-9771-6f11080f65dc None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:16.917288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-99b63e76-81fb-4105-9771-6f11080f65dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:16.917288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:16.917288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:16.917288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:16.917288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:16.917288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:16.917288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:16.917288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:16.917288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:16.917288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:16.917288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:16.917288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:16.917288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:16.917288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:16.917288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:16.917288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:16.919510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:16.919510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:16.919510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:16.919510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:16.919510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:16.919510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:16.919510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:16.919510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:16.919510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:16.919510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:16.919510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:16.919510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:16.919510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:16.919510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:16.919510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:16.919510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:16.919510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:16.921622 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:16.921622 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:16.921622 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:16.930889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 323/654] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:16.932788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b63e76-81fb-4105-9771-6f11080f65dc None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:16.980640 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d6e60-0fe7-4a25-9930-fa9841727bcc None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:17.045805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60dcfe-0167-482d-a8c3-df07515d7616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:17.046821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60dcfe-0167-482d-a8c3-df07515d7616 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:17.047440 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60dcfe-0167-482d-a8c3-df07515d7616 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:17.047937 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60dcfe-0167-482d-a8c3-df07515d7616 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:17.069516 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60dcfe-0167-482d-a8c3-df07515d7616 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:17.070784 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60dcfe-0167-482d-a8c3-df07515d7616 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:17.096939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-0b60dcfe-0167-482d-a8c3-df07515d7616 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:17.096939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:17.096939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:17.096939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:17.096939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:17.096939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:17.096939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:17.096939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:17.096939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:17.096939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:17.096939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:17.096939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:17.096939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:17.096939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:17.096939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:17.096939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:17.098306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:17.098306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:17.098306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:17.098306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:17.098306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:17.098306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:17.098306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:17.098306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:17.098306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:17.098306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:17.098306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:17.098306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:17.098306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:17.098306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:17.098306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:17.098306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:17.098306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:17.099385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:17.099385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:17.099385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:17.104604 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 332/655] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:17.106335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60dcfe-0167-482d-a8c3-df07515d7616 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:17.129427 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b63e76-81fb-4105-9771-6f11080f65dc None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:17.145263 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:17.182516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464220c-eede-47a8-8e87-f1aae220129a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:17.184101 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464220c-eede-47a8-8e87-f1aae220129a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:17.184886 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464220c-eede-47a8-8e87-f1aae220129a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:17.185644 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464220c-eede-47a8-8e87-f1aae220129a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:17.200621 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464220c-eede-47a8-8e87-f1aae220129a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:17.204591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464220c-eede-47a8-8e87-f1aae220129a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:17.229139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-a464220c-eede-47a8-8e87-f1aae220129a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:31:17.229139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:17.229139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:17.229139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:17.229139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:17.229139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:17.229139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:17.229139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:17.229139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:17.229139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:17.229139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:17.229139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:17.229139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:17.229139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:17.229139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:17.229139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:17.230551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:17.232718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:17.232718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:17.232718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:17.232718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:17.232718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:17.232718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:17.232718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:17.232718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:17.232718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:31:17.232718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:17.237084 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 324/656] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:31:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:17.238938 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464220c-eede-47a8-8e87-f1aae220129a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:17.254445 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:17.262547 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60dcfe-0167-482d-a8c3-df07515d7616 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:17.313725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be642a94-5e0a-4618-b436-83aacaccfd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:17.314249 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be642a94-5e0a-4618-b436-83aacaccfd81 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:17.314525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be642a94-5e0a-4618-b436-83aacaccfd81 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:17.314791 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be642a94-5e0a-4618-b436-83aacaccfd81 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:17.315456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be642a94-5e0a-4618-b436-83aacaccfd81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:17.315752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be642a94-5e0a-4618-b436-83aacaccfd81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:17.317378 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be642a94-5e0a-4618-b436-83aacaccfd81 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:17.340125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 333/657] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:17.340125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be642a94-5e0a-4618-b436-83aacaccfd81 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:17.447180 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464220c-eede-47a8-8e87-f1aae220129a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:17.506458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608f230-5d36-4d16-b50c-8d42ca435b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:17.508687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608f230-5d36-4d16-b50c-8d42ca435b4a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:17.509452 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608f230-5d36-4d16-b50c-8d42ca435b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:17.510149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608f230-5d36-4d16-b50c-8d42ca435b4a None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:17.511634 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608f230-5d36-4d16-b50c-8d42ca435b4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:17.512457 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608f230-5d36-4d16-b50c-8d42ca435b4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:17.516579 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608f230-5d36-4d16-b50c-8d42ca435b4a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:17.539825 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be642a94-5e0a-4618-b436-83aacaccfd81 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:17.546634 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 325/658] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:17.548076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608f230-5d36-4d16-b50c-8d42ca435b4a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:17.605362 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:17.617521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7cf0f-a269-44cd-afee-2e80438642d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:17.618313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7cf0f-a269-44cd-afee-2e80438642d4 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:17.618692 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7cf0f-a269-44cd-afee-2e80438642d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:17.619065 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7cf0f-a269-44cd-afee-2e80438642d4 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:17.619929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7cf0f-a269-44cd-afee-2e80438642d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:17.620390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7cf0f-a269-44cd-afee-2e80438642d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:17.622625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7cf0f-a269-44cd-afee-2e80438642d4 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:17.648138 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 334/659] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Aug 30 14:31:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:17.649199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7cf0f-a269-44cd-afee-2e80438642d4 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:17.680422 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:17.725959 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608f230-5d36-4d16-b50c-8d42ca435b4a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:17.769893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3a6eb-5ee0-442a-a41d-b28c26332001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:17.770833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3a6eb-5ee0-442a-a41d-b28c26332001 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:17.771095 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3a6eb-5ee0-442a-a41d-b28c26332001 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:17.771493 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3a6eb-5ee0-442a-a41d-b28c26332001 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:17.779764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3a6eb-5ee0-442a-a41d-b28c26332001 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:17.782435 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3a6eb-5ee0-442a-a41d-b28c26332001 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:17.796073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-29b3a6eb-5ee0-442a-a41d-b28c26332001 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:17.796073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:17.796073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:17.796073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:17.796073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:17.796073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:17.796073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:17.796073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:17.796073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:17.796073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:17.796073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:17.796073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:17.796073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:17.796073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:17.796073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:17.797897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:17.797897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:17.797897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:17.797897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:17.797897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:17.797897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:17.797897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:17.797897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:17.797897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:17.797897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:17.797897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:17.797897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:17.797897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:17.797897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:17.797897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:17.797897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:17.797897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:17.799733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:17.799733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:17.799733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:17.799733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:17.799733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:17.799733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:17.799733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:17.799733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:17.804625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 326/660] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:17.805732 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3a6eb-5ee0-442a-a41d-b28c26332001 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:17.860480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7cf0f-a269-44cd-afee-2e80438642d4 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:17.922377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:17.923620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a464a-4e36-42a1-b892-5f4622af8fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:17.924273 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a464a-4e36-42a1-b892-5f4622af8fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:17.924549 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a464a-4e36-42a1-b892-5f4622af8fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:17.924821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a464a-4e36-42a1-b892-5f4622af8fc2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:17.934852 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a464a-4e36-42a1-b892-5f4622af8fc2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:17.936182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3a6eb-5ee0-442a-a41d-b28c26332001 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:17.936901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a464a-4e36-42a1-b892-5f4622af8fc2 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:17.951035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-f74a464a-4e36-42a1-b892-5f4622af8fc2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:31:17.951035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:17.951035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:17.951035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:17.951035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:17.951035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:17.951035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:17.951035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:17.951035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:17.951035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:17.951035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:17.951035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:17.951035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:17.951035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:17.951035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:17.951035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:17.953734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:17.955637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:17.955637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:17.955637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:17.955637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:17.955637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:17.955637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:17.955637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:17.955637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:17.955637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:31:17.955637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:17.955637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 335/661] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:31:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:17.955637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a464a-4e36-42a1-b892-5f4622af8fc2 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:18.007591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287f80-44bd-4a06-9dbe-5fe23735f1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:18.009241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287f80-44bd-4a06-9dbe-5fe23735f1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:18.009862 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287f80-44bd-4a06-9dbe-5fe23735f1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:18.010574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287f80-44bd-4a06-9dbe-5fe23735f1a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:18.061024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:18.113557 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a464a-4e36-42a1-b892-5f4622af8fc2 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:18.116112 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df287f80-44bd-4a06-9dbe-5fe23735f1a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:18.117937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df287f80-44bd-4a06-9dbe-5fe23735f1a8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:18.167042 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab71f0-56c8-48a6-87b7-48d684340e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:18.167978 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab71f0-56c8-48a6-87b7-48d684340e08 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:18.168419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab71f0-56c8-48a6-87b7-48d684340e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:18.168806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab71f0-56c8-48a6-87b7-48d684340e08 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:18.169652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab71f0-56c8-48a6-87b7-48d684340e08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:18.170158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab71f0-56c8-48a6-87b7-48d684340e08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:18.172516 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab71f0-56c8-48a6-87b7-48d684340e08 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:18.195655 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 336/662] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:18.195655 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95017143-ea89-4a19-9012-0eb189f11b08 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:18.195655 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95017143-ea89-4a19-9012-0eb189f11b08 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:18.195655 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95017143-ea89-4a19-9012-0eb189f11b08 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:18.195655 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 337/663] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:18.195655 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95017143-ea89-4a19-9012-0eb189f11b08 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:18.274020 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:18.285665 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 327/664] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:31:18.286551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287f80-44bd-4a06-9dbe-5fe23735f1a8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:18.361996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95017143-ea89-4a19-9012-0eb189f11b08 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:18.404758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d31c1c-bbba-4996-9b5d-fb8762c022e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:18.405407 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d31c1c-bbba-4996-9b5d-fb8762c022e4 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:18.405741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d31c1c-bbba-4996-9b5d-fb8762c022e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:18.406068 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d31c1c-bbba-4996-9b5d-fb8762c022e4 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:18.406951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d31c1c-bbba-4996-9b5d-fb8762c022e4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:18.407391 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d31c1c-bbba-4996-9b5d-fb8762c022e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:18.409173 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d31c1c-bbba-4996-9b5d-fb8762c022e4 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:18.411459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:18.419027 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287f80-44bd-4a06-9dbe-5fe23735f1a8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:18.434926 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 338/665] 10.222.0.25 () {56 vars in 1139 bytes} [Tue Aug 30 14:31:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:18.437164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63062fbb-d799-4de8-b9c5-eba2146a00c3 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:18.437605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63062fbb-d799-4de8-b9c5-eba2146a00c3 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:18.437968 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63062fbb-d799-4de8-b9c5-eba2146a00c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:18.458722 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-63062fbb-d799-4de8-b9c5-eba2146a00c3 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:18.460019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-63062fbb-d799-4de8-b9c5-eba2146a00c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:18.475477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d7417-c0e8-47fd-b8cf-5addf88c0080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:18.476477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d7417-c0e8-47fd-b8cf-5addf88c0080 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:18.476842 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d7417-c0e8-47fd-b8cf-5addf88c0080 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:18.477270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d7417-c0e8-47fd-b8cf-5addf88c0080 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:18.478114 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d7417-c0e8-47fd-b8cf-5addf88c0080 None admin] RBAC: Adding query filter params (role.id=f0fd911d7c9246aeb3d67299473a9f42, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=2788fece401a4a7ebdb516afe2dd8ec8) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:18.478423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d7417-c0e8-47fd-b8cf-5addf88c0080 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:18.480250 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d7417-c0e8-47fd-b8cf-5addf88c0080 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:18.487675 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 328/666] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:18 2022] GET /identity/v3/role_assignments?user.id=2788fece401a4a7ebdb516afe2dd8ec8&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=f0fd911d7c9246aeb3d67299473a9f42 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:18.489844 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64bf90-f08d-471c-86ff-18a9689bcec8 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:18.490310 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64bf90-f08d-471c-86ff-18a9689bcec8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:18.490796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64bf90-f08d-471c-86ff-18a9689bcec8 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:18.493496 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 329/667] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:18.494822 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64bf90-f08d-471c-86ff-18a9689bcec8 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:18.573381 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 339/668] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:31:18.575103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63062fbb-d799-4de8-b9c5-eba2146a00c3 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:18.619717 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:18.723705 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64bf90-f08d-471c-86ff-18a9689bcec8 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:18.790949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b303238-2249-433f-a5f4-8b685c2f5e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:18.791601 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b303238-2249-433f-a5f4-8b685c2f5e5f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:18.791917 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b303238-2249-433f-a5f4-8b685c2f5e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:18.792322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b303238-2249-433f-a5f4-8b685c2f5e5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:18.799326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63062fbb-d799-4de8-b9c5-eba2146a00c3 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:18.805318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b303238-2249-433f-a5f4-8b685c2f5e5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:18.806789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b303238-2249-433f-a5f4-8b685c2f5e5f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:18.823908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-8b303238-2249-433f-a5f4-8b685c2f5e5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:18.823908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:18.823908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:18.823908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:18.823908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:18.823908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:18.823908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:18.823908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:18.823908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:18.823908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:18.823908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:18.823908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:18.823908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:18.823908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:18.823908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:18.823908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:18.825995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:18.825995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:18.825995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:18.825995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:18.825995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:18.825995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:18.825995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:18.825995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:18.825995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:18.825995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:18.825995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:18.825995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:18.825995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:18.825995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:18.825995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:18.825995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:18.825995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:18.827866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:18.827866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:18.827866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:18.827866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 330/669] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:18.827866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ad79a-6f43-43be-93dc-0a7220bcb372 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:18.827866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ad79a-6f43-43be-93dc-0a7220bcb372 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:18.827866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ad79a-6f43-43be-93dc-0a7220bcb372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:18.850390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:18.852341 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-057ad79a-6f43-43be-93dc-0a7220bcb372 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:18.854742 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-057ad79a-6f43-43be-93dc-0a7220bcb372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:18.873013 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74648fe-1d4c-4e4a-a6d4-de697f5a00b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:18.874223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74648fe-1d4c-4e4a-a6d4-de697f5a00b4 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:18.874728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74648fe-1d4c-4e4a-a6d4-de697f5a00b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:18.875207 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74648fe-1d4c-4e4a-a6d4-de697f5a00b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:18.976978 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 331/670] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:31:18.978788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ad79a-6f43-43be-93dc-0a7220bcb372 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:19.035567 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74648fe-1d4c-4e4a-a6d4-de697f5a00b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:19.038331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74648fe-1d4c-4e4a-a6d4-de697f5a00b4 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:19.044759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:19.166933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ad79a-6f43-43be-93dc-0a7220bcb372 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:19.230212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a7a32-4f06-4932-89a2-714424524294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:19.231624 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a7a32-4f06-4932-89a2-714424524294 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:19.232405 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a7a32-4f06-4932-89a2-714424524294 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:19.233174 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a7a32-4f06-4932-89a2-714424524294 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:19.234631 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a7a32-4f06-4932-89a2-714424524294 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:19.235406 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a7a32-4f06-4932-89a2-714424524294 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:19.238259 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a7a32-4f06-4932-89a2-714424524294 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:19.255553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 340/671] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:31:19.257511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74648fe-1d4c-4e4a-a6d4-de697f5a00b4 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:19.262094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 332/672] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:19.263878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a7a32-4f06-4932-89a2-714424524294 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:19.320622 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:19.440545 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:19.452428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a7a32-4f06-4932-89a2-714424524294 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:19.484722 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74648fe-1d4c-4e4a-a6d4-de697f5a00b4 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:19.513375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f52c4-d816-4423-89ce-ee10371621f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:19.514972 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f52c4-d816-4423-89ce-ee10371621f5 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:19.515736 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f52c4-d816-4423-89ce-ee10371621f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:19.516537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f52c4-d816-4423-89ce-ee10371621f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:19.536721 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f52c4-d816-4423-89ce-ee10371621f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:19.538602 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f52c4-d816-4423-89ce-ee10371621f5 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:19.555300 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e4b12-6081-4d7c-a65b-fa8088f3bdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:19.556390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e4b12-6081-4d7c-a65b-fa8088f3bdb7 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:19.556859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e4b12-6081-4d7c-a65b-fa8088f3bdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:19.557197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-626f52c4-d816-4423-89ce-ee10371621f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:19.557197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:19.557197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:19.557197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:19.557197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:19.557197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:19.557197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:19.557197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:19.557197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:19.557197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:19.557197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:19.557197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:19.557197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:19.557197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:19.557197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:19.557197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:19.558994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:19.558994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:19.558994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:19.558994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:19.558994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:19.558994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:19.558994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:19.558994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:19.558994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:19.558994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:19.558994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:19.558994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:19.558994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:19.558994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:19.558994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:19.558994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:19.558994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:19.560829 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:19.560829 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:19.560829 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:19.561223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e4b12-6081-4d7c-a65b-fa8088f3bdb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:19.563248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 333/673] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:19.564738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f52c4-d816-4423-89ce-ee10371621f5 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:19.581288 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e4b12-6081-4d7c-a65b-fa8088f3bdb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:19.591942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-634e4b12-6081-4d7c-a65b-fa8088f3bdb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:19.591942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:19.591942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:19.591942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:19.591942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:19.591942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:19.591942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:19.591942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:19.591942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:19.591942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:19.591942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:19.591942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:19.591942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:19.591942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:19.591942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:19.591942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:19.593223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:19.593223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:19.593223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:19.593223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:19.593223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:19.593223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:19.593223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:19.593223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:19.593223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:19.593223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:19.593223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:19.593223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:19.593223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:19.593223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:19.593223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:19.593223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:19.594089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:19.594089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:19.594089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:19.594089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:19.594089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:19.594089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:19.594089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:19.594089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:19.594089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:19.594089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:19.594089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:19.594089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:19.600136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 341/674] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:19.600136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e4b12-6081-4d7c-a65b-fa8088f3bdb7 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:19.744091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f52c4-d816-4423-89ce-ee10371621f5 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:19.749799 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:19.786096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e4b12-6081-4d7c-a65b-fa8088f3bdb7 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:19.797578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f883f8-4a8d-4486-aad0-128187292911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:19.798710 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f883f8-4a8d-4486-aad0-128187292911 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:19.799191 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f883f8-4a8d-4486-aad0-128187292911 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:19.799774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f883f8-4a8d-4486-aad0-128187292911 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:19.816023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f883f8-4a8d-4486-aad0-128187292911 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:19.819900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f883f8-4a8d-4486-aad0-128187292911 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:19.830248 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f0838-e3ec-497d-932a-1856f7274d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:19.831062 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f0838-e3ec-497d-932a-1856f7274d50 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:19.831464 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f0838-e3ec-497d-932a-1856f7274d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:19.831882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f0838-e3ec-497d-932a-1856f7274d50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:19.845054 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f0838-e3ec-497d-932a-1856f7274d50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:19.848553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-13f883f8-4a8d-4486-aad0-128187292911 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:19.848553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:19.848553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:19.848553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:19.848553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:19.848553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:19.848553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:19.848553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:19.848553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:19.848553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:19.848553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:19.848553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:19.848553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:19.848553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:19.848553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:19.850189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:19.850189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:19.850189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:19.850189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:19.850189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:19.850189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:19.850189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:19.850189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:19.850189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:19.850189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:19.850189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:19.850189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:19.850189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:19.850189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:19.850189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:19.850189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:19.850189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:19.860567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:19.860567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:19.860567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:19.860567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:19.860567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:19.860567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:19.860567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:19.860567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:19.860567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 334/675] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:19.862185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-da0f0838-e3ec-497d-932a-1856f7274d50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:19.862185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:19.862185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:19.862185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:19.862185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:19.862185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:19.862185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:19.862185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:19.862185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:19.862185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:19.862185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:19.862185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:19.862185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:19.862185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:19.862185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:19.862185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:19.864434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:19.864434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:19.864434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:19.864434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:19.864434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:19.864434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:19.864434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:19.864434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:19.864434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:19.864434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:19.864434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:19.864434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:19.864434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:19.864434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:19.864434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:19.864434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:19.867041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:19.867041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:19.867041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:19.867041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:19.867041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:19.867041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:19.867041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:19.867041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:19.867041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:19.867041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:19.867041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:19.867041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:19.869808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f883f8-4a8d-4486-aad0-128187292911 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:19.869989 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 342/676] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:31:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:19.869989 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f0838-e3ec-497d-932a-1856f7274d50 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:19.886789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:20.025853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f883f8-4a8d-4486-aad0-128187292911 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:20.038412 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f0838-e3ec-497d-932a-1856f7274d50 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:20.086243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cd05f-ea19-4c70-9cc5-e9e18bdb4524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:20.087207 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cd05f-ea19-4c70-9cc5-e9e18bdb4524 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:20.087652 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cd05f-ea19-4c70-9cc5-e9e18bdb4524 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:20.088092 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cd05f-ea19-4c70-9cc5-e9e18bdb4524 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:20.089221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78cd05f-ea19-4c70-9cc5-e9e18bdb4524 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:20.089730 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78cd05f-ea19-4c70-9cc5-e9e18bdb4524 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:20.091844 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78cd05f-ea19-4c70-9cc5-e9e18bdb4524 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:20.104300 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847af304-2e48-45ec-9a3a-c82d1e443e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:20.105504 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847af304-2e48-45ec-9a3a-c82d1e443e09 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:20.105901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847af304-2e48-45ec-9a3a-c82d1e443e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:20.106352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847af304-2e48-45ec-9a3a-c82d1e443e09 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:20.107501 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847af304-2e48-45ec-9a3a-c82d1e443e09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:20.108112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847af304-2e48-45ec-9a3a-c82d1e443e09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:20.110100 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847af304-2e48-45ec-9a3a-c82d1e443e09 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:20.122488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 335/677] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:20.122488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cd05f-ea19-4c70-9cc5-e9e18bdb4524 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:20.133012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 343/678] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:31:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:20.133012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847af304-2e48-45ec-9a3a-c82d1e443e09 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:20.142396 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:20.233114 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:20.295172 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cd05f-ea19-4c70-9cc5-e9e18bdb4524 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:20.316081 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847af304-2e48-45ec-9a3a-c82d1e443e09 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:20.358159 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7282bc-5c65-4239-8383-bda2901bc359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:20.359294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a5652-4434-4302-924b-f784fc149ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:20.359294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a5652-4434-4302-924b-f784fc149ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:20.359578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7282bc-5c65-4239-8383-bda2901bc359 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:20.359757 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a5652-4434-4302-924b-f784fc149ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:20.359898 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7282bc-5c65-4239-8383-bda2901bc359 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:20.360060 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a5652-4434-4302-924b-f784fc149ba5 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:20.360182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7282bc-5c65-4239-8383-bda2901bc359 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:20.360302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a5652-4434-4302-924b-f784fc149ba5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:20.360785 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a5652-4434-4302-924b-f784fc149ba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:20.361222 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7282bc-5c65-4239-8383-bda2901bc359 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:20.361755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7282bc-5c65-4239-8383-bda2901bc359 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:20.362016 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a5652-4434-4302-924b-f784fc149ba5 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:20.364467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7282bc-5c65-4239-8383-bda2901bc359 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:20.381208 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 344/679] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:31:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:20.381208 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a5652-4434-4302-924b-f784fc149ba5 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:20.395911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 336/680] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:20.395911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7282bc-5c65-4239-8383-bda2901bc359 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:20.516039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:20.551861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7282bc-5c65-4239-8383-bda2901bc359 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:20.577003 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a5652-4434-4302-924b-f784fc149ba5 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:20.610302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:20.624779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c771f22-4707-402d-bc01-a30e450b1914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:20.625549 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c771f22-4707-402d-bc01-a30e450b1914 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:20.625890 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c771f22-4707-402d-bc01-a30e450b1914 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:20.626225 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c771f22-4707-402d-bc01-a30e450b1914 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:20.627595 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e3062-a1b5-4844-8798-e2d05bc7ed88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:20.628287 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e3062-a1b5-4844-8798-e2d05bc7ed88 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:20.628560 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e3062-a1b5-4844-8798-e2d05bc7ed88 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:20.628826 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e3062-a1b5-4844-8798-e2d05bc7ed88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:20.636188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c771f22-4707-402d-bc01-a30e450b1914 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:20.637668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c771f22-4707-402d-bc01-a30e450b1914 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:20.641356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e3062-a1b5-4844-8798-e2d05bc7ed88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:20.642844 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e3062-a1b5-4844-8798-e2d05bc7ed88 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:20.650257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-2c771f22-4707-402d-bc01-a30e450b1914 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:20.650257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:20.650257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:20.650257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:20.650257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:20.650257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:20.650257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:20.650257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:20.650257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:20.650257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:20.650257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:20.650257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:20.650257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:20.650257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:20.650257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:20.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:20.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:20.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:20.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:20.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:20.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:20.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:20.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:20.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:20.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:20.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:20.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:20.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:20.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:20.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:20.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:20.652075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:20.653648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:20.653648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:20.653648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:20.653648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:20.653648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:20.653648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:20.653648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:20.653648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:20.653648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 345/681] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:20.653648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c771f22-4707-402d-bc01-a30e450b1914 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:20.666489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-7b8e3062-a1b5-4844-8798-e2d05bc7ed88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:20.666489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:20.666489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:20.666489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:20.666489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:20.666489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:20.666489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:20.666489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:20.666489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:20.666489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:20.666489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:20.666489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:20.666489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:20.666489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:20.666489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:20.666489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:20.667589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:20.667589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:20.667589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:20.667589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:20.667589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:20.667589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:20.667589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:20.667589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:20.667589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:20.667589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:20.667589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:20.667589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:20.667589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:20.667589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:20.667589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:20.667589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:20.667589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:20.668683 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:20.668683 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:20.668683 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:20.668683 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 337/682] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:20.668683 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e3062-a1b5-4844-8798-e2d05bc7ed88 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:20.777022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c771f22-4707-402d-bc01-a30e450b1914 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:20.802854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e3062-a1b5-4844-8798-e2d05bc7ed88 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:20.806185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:20.817777 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cddaec-8130-4ff7-8fd0-e7289bb49413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:20.818438 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cddaec-8130-4ff7-8fd0-e7289bb49413 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:20.818716 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cddaec-8130-4ff7-8fd0-e7289bb49413 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:20.818984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cddaec-8130-4ff7-8fd0-e7289bb49413 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:20.833045 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cddaec-8130-4ff7-8fd0-e7289bb49413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:20.834972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cddaec-8130-4ff7-8fd0-e7289bb49413 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:20.843862 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f72054-b4ff-41fe-bd76-2044de0364ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:20.844636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f72054-b4ff-41fe-bd76-2044de0364ee None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:20.845420 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f72054-b4ff-41fe-bd76-2044de0364ee None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:20.845786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f72054-b4ff-41fe-bd76-2044de0364ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:20.846778 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f72054-b4ff-41fe-bd76-2044de0364ee None admin] RBAC: Adding query filter params (role.id=f0fd911d7c9246aeb3d67299473a9f42, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=69de9368993f402cb6325e5022e064d4) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:20.847170 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f72054-b4ff-41fe-bd76-2044de0364ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:20.848795 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f72054-b4ff-41fe-bd76-2044de0364ee None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:20.851459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-53cddaec-8130-4ff7-8fd0-e7289bb49413 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:20.851459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:20.851459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:20.851459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:20.851459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:20.851459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:20.851459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:20.851459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:20.851459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:20.851459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:20.851459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:20.851459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:20.851459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:20.851459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:20.851459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:20.851459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:20.853331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:20.853331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:20.853331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:20.853331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:20.853331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:20.853331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:20.853331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:20.853331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:20.853331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:20.853331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:20.853331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:20.853331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:20.853331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:20.853331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:20.853331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:20.853331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:20.853331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:20.855071 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:20.855071 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:20.855071 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:20.855351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 346/683] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:20.865029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 338/684] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:20 2022] GET /identity/v3/role_assignments?user.id=69de9368993f402cb6325e5022e064d4&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=f0fd911d7c9246aeb3d67299473a9f42 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:20.865302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e8c7b-b016-42e3-a0a5-a4a937794ecc None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:20.865443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f72054-b4ff-41fe-bd76-2044de0364ee None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:20.865576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e8c7b-b016-42e3-a0a5-a4a937794ecc None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:20.865576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e8c7b-b016-42e3-a0a5-a4a937794ecc None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:20.865576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 347/685] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:20.865576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e8c7b-b016-42e3-a0a5-a4a937794ecc None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:20.958141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:21.030238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e8c7b-b016-42e3-a0a5-a4a937794ecc None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:21.050947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f72054-b4ff-41fe-bd76-2044de0364ee None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:21.085434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51a6a8-a419-4df8-946e-d4e2b3712633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:21.086369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51a6a8-a419-4df8-946e-d4e2b3712633 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:21.086852 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51a6a8-a419-4df8-946e-d4e2b3712633 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:21.087334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51a6a8-a419-4df8-946e-d4e2b3712633 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:21.088564 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51a6a8-a419-4df8-946e-d4e2b3712633 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:21.089047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51a6a8-a419-4df8-946e-d4e2b3712633 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:21.091555 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51a6a8-a419-4df8-946e-d4e2b3712633 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:21.098439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cd5cc-7a20-4bac-bca6-123e95d5d0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:21.099462 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd5cc-7a20-4bac-bca6-123e95d5d0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:21.099929 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd5cc-7a20-4bac-bca6-123e95d5d0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:21.100424 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd5cc-7a20-4bac-bca6-123e95d5d0a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:21.101682 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4cd5cc-7a20-4bac-bca6-123e95d5d0a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:21.102207 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4cd5cc-7a20-4bac-bca6-123e95d5d0a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:21.105099 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4cd5cc-7a20-4bac-bca6-123e95d5d0a6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:21.115023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 348/686] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:21.116639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51a6a8-a419-4df8-946e-d4e2b3712633 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:21.128883 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 339/687] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:21.131129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da8eb0-5a86-4a34-9a80-2fb769c5a1df None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:21.131410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da8eb0-5a86-4a34-9a80-2fb769c5a1df None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:21.131934 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da8eb0-5a86-4a34-9a80-2fb769c5a1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:21.157011 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d9da8eb0-5a86-4a34-9a80-2fb769c5a1df None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:21.158281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d9da8eb0-5a86-4a34-9a80-2fb769c5a1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:21.178702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:21.258501 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51a6a8-a419-4df8-946e-d4e2b3712633 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:21.268045 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 340/688] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:31:21.268045 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da8eb0-5a86-4a34-9a80-2fb769c5a1df None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:21.304104 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0664d-3598-41a3-b130-0bcad2ad9cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:21.304843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0664d-3598-41a3-b130-0bcad2ad9cbe None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:21.305115 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0664d-3598-41a3-b130-0bcad2ad9cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:21.305382 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0664d-3598-41a3-b130-0bcad2ad9cbe None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:21.306047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0664d-3598-41a3-b130-0bcad2ad9cbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:21.306339 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0664d-3598-41a3-b130-0bcad2ad9cbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:21.307747 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0664d-3598-41a3-b130-0bcad2ad9cbe None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:21.328050 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 349/689] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:21.328050 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0664d-3598-41a3-b130-0bcad2ad9cbe None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:21.335988 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:21.425671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da8eb0-5a86-4a34-9a80-2fb769c5a1df None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:21.476558 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c078c-c08f-489c-ba62-867c3b030546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:21.477405 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c078c-c08f-489c-ba62-867c3b030546 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:21.477772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c078c-c08f-489c-ba62-867c3b030546 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:21.478140 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c078c-c08f-489c-ba62-867c3b030546 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:21.488985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c078c-c08f-489c-ba62-867c3b030546 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:21.490870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c078c-c08f-489c-ba62-867c3b030546 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:21.511413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-aa8c078c-c08f-489c-ba62-867c3b030546 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:31:21.511413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:21.511413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:21.511413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:21.511413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:21.511413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:21.511413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:21.511413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:21.511413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:21.511413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:21.511413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:21.511413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:21.511413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:21.511413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:21.511413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:21.511413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:21.513509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:21.515450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:21.515450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:21.515450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:21.515450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:21.515450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:21.515450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:21.515450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:21.515450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:21.515450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:31:21.515450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:21.519917 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 341/690] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Aug 30 14:31:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:21.521041 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c078c-c08f-489c-ba62-867c3b030546 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:21.565350 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:21.569528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0664d-3598-41a3-b130-0bcad2ad9cbe None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:21.617757 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d56bd-edea-4258-9960-d1b28672b845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:21.619120 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d56bd-edea-4258-9960-d1b28672b845 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:21.619516 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d56bd-edea-4258-9960-d1b28672b845 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:21.619924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d56bd-edea-4258-9960-d1b28672b845 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:21.620861 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d56bd-edea-4258-9960-d1b28672b845 None admin] RBAC: Adding query filter params (role.id=f0fd911d7c9246aeb3d67299473a9f42, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=932fce741c5a4deba79a310a14e6bd2c) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:21.621302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d56bd-edea-4258-9960-d1b28672b845 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:21.622963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d56bd-edea-4258-9960-d1b28672b845 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:21.637786 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 350/691] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:21 2022] GET /identity/v3/role_assignments?user.id=932fce741c5a4deba79a310a14e6bd2c&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=f0fd911d7c9246aeb3d67299473a9f42 => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:21.638502 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d56bd-edea-4258-9960-d1b28672b845 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:21.662894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:21.677052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c078c-c08f-489c-ba62-867c3b030546 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:21.768572 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4b519-2fee-4c27-99de-18245e4ee6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:21.770223 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4b519-2fee-4c27-99de-18245e4ee6ae None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:21.770694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4b519-2fee-4c27-99de-18245e4ee6ae None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:21.771123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4b519-2fee-4c27-99de-18245e4ee6ae None admin] PATH_INFO: `/v3/users/swift` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:21.773634 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d56bd-edea-4258-9960-d1b28672b845 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:21.790289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4b519-2fee-4c27-99de-18245e4ee6ae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:21.794549 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4b519-2fee-4c27-99de-18245e4ee6ae None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:21.816125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba12f4b-0049-4195-aadf-e55e98ff0fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:21.817121 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba12f4b-0049-4195-aadf-e55e98ff0fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:21.817592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba12f4b-0049-4195-aadf-e55e98ff0fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:21.818070 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba12f4b-0049-4195-aadf-e55e98ff0fb8 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:21.819111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba12f4b-0049-4195-aadf-e55e98ff0fb8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:21.819608 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba12f4b-0049-4195-aadf-e55e98ff0fb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:21.821436 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba12f4b-0049-4195-aadf-e55e98ff0fb8 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:21.826342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-28c4b519-2fee-4c27-99de-18245e4ee6ae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:31:21.826342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:21.826342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:21.826342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:21.826342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:21.826342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:21.826342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:21.826342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:21.826342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:21.826342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:21.826342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:21.826342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:21.826342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:21.826342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:21.826342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:21.826342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:21.827444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:21.828846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:21.828846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:21.828846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:21.828846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:21.828846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:21.828846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:21.828846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:21.828846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:21.828846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:31:21.828846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:21.834770 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 342/692] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:21.836823 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4b519-2fee-4c27-99de-18245e4ee6ae None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:21.849928 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 351/693] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Aug 30 14:31:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:21.851239 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba12f4b-0049-4195-aadf-e55e98ff0fb8 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:21.879229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:22.034428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba12f4b-0049-4195-aadf-e55e98ff0fb8 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:22.037351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4b519-2fee-4c27-99de-18245e4ee6ae None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:22.098642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924eb3d-c276-4851-a44b-8ea25fe2fa6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:22.099737 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924eb3d-c276-4851-a44b-8ea25fe2fa6e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:22.100255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924eb3d-c276-4851-a44b-8ea25fe2fa6e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:22.100714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924eb3d-c276-4851-a44b-8ea25fe2fa6e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:22.101804 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924eb3d-c276-4851-a44b-8ea25fe2fa6e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:22.104429 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924eb3d-c276-4851-a44b-8ea25fe2fa6e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:22.107826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1992b8e-bc69-4ea1-916f-c9586a39fdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:22.109496 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1992b8e-bc69-4ea1-916f-c9586a39fdec None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:22.110153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1992b8e-bc69-4ea1-916f-c9586a39fdec None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:22.111247 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1992b8e-bc69-4ea1-916f-c9586a39fdec None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:22.113058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1992b8e-bc69-4ea1-916f-c9586a39fdec None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:22.113936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1992b8e-bc69-4ea1-916f-c9586a39fdec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:22.116857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1992b8e-bc69-4ea1-916f-c9586a39fdec None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:22.126397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-5924eb3d-c276-4851-a44b-8ea25fe2fa6e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:31:22.126397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:22.126397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:31:22.126397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:31:22.126397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:31:22.126397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:31:22.126397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:31:22.126397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:22.126397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:31:22.126397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:22.126397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:22.126397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:22.126397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:22.126397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:22.126397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:22.126397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:22.126397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:22.128574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:22.128574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:22.128574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:22.128574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:22.128574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:22.128574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:22.128574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:31:22.128574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:31:22.128574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:31:22.128574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:31:22.128574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:22.128574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:22.128574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:22.128574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:22.128574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:22.128574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:22.130459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:22.130459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:22.130459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:22.130459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:22.130459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:22.130459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:22.130459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:22.130459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:22.130459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:22.130459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:22.130459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:31:22.130459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:22.130459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:31:22.130459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:22.130459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 343/694] 10.222.0.25 () {56 vars in 1129 bytes} [Tue Aug 30 14:31:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:22.132570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924eb3d-c276-4851-a44b-8ea25fe2fa6e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:22.176190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 352/695] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Aug 30 14:31:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:22.176190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1992b8e-bc69-4ea1-916f-c9586a39fdec None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:22.182362 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:22.339193 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:22.353297 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924eb3d-c276-4851-a44b-8ea25fe2fa6e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:22.394126 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1992b8e-bc69-4ea1-916f-c9586a39fdec None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:22.422876 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6182c-2ee6-46cb-ba8d-b2639c914afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:22.423943 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6182c-2ee6-46cb-ba8d-b2639c914afb None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:22.424492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6182c-2ee6-46cb-ba8d-b2639c914afb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:22.424969 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6182c-2ee6-46cb-ba8d-b2639c914afb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:22.474891 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006bea8-739b-4a44-89f4-1f163fc83770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:22.476357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006bea8-739b-4a44-89f4-1f163fc83770 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:22.477000 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006bea8-739b-4a44-89f4-1f163fc83770 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:22.477749 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006bea8-739b-4a44-89f4-1f163fc83770 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:22.479032 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006bea8-739b-4a44-89f4-1f163fc83770 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:22.479740 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006bea8-739b-4a44-89f4-1f163fc83770 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:22.482662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006bea8-739b-4a44-89f4-1f163fc83770 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:22.491690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 353/696] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Aug 30 14:31:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:22.493605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006bea8-739b-4a44-89f4-1f163fc83770 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:22.553315 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6182c-2ee6-46cb-ba8d-b2639c914afb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:22.555842 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6182c-2ee6-46cb-ba8d-b2639c914afb None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:22.587507 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:22.680629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006bea8-739b-4a44-89f4-1f163fc83770 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:22.736475 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 344/697] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:31:22.738003 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6182c-2ee6-46cb-ba8d-b2639c914afb None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:22.745884 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72827a60-3fcb-4c95-a3f6-410005d6013f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:22.747144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72827a60-3fcb-4c95-a3f6-410005d6013f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:22.747804 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72827a60-3fcb-4c95-a3f6-410005d6013f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:22.748487 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72827a60-3fcb-4c95-a3f6-410005d6013f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:22.772579 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:22.878142 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72827a60-3fcb-4c95-a3f6-410005d6013f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:22.881355 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72827a60-3fcb-4c95-a3f6-410005d6013f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:22.914146 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6182c-2ee6-46cb-ba8d-b2639c914afb None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:22.946177 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:22.975805 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31b503-b455-4bf3-b05f-b60ef7d22c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:22.976818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31b503-b455-4bf3-b05f-b60ef7d22c1c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:22.977271 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31b503-b455-4bf3-b05f-b60ef7d22c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:22.977722 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31b503-b455-4bf3-b05f-b60ef7d22c1c None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:22.978831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31b503-b455-4bf3-b05f-b60ef7d22c1c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:22.979300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31b503-b455-4bf3-b05f-b60ef7d22c1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:22.981755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31b503-b455-4bf3-b05f-b60ef7d22c1c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:22.993035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 345/698] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Aug 30 14:31:22 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:22.994531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31b503-b455-4bf3-b05f-b60ef7d22c1c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:23.083551 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 354/699] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:31:23.094196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72827a60-3fcb-4c95-a3f6-410005d6013f None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:23.177855 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:23.183526 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31b503-b455-4bf3-b05f-b60ef7d22c1c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:23.246557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491a506-287b-41fd-9673-c0667e0b4e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:23.247456 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491a506-287b-41fd-9673-c0667e0b4e8a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:23.247912 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491a506-287b-41fd-9673-c0667e0b4e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:23.248994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491a506-287b-41fd-9673-c0667e0b4e8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:23.273797 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491a506-287b-41fd-9673-c0667e0b4e8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:23.275956 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491a506-287b-41fd-9673-c0667e0b4e8a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:23.281302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72827a60-3fcb-4c95-a3f6-410005d6013f None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:23.303893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-8491a506-287b-41fd-9673-c0667e0b4e8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:23.303893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:23.303893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:23.303893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:23.303893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:23.303893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:23.303893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:23.303893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:23.303893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:23.303893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:23.303893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:23.303893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:23.303893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:23.303893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:23.303893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:23.303893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:23.305047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:23.305047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:23.305047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:23.305047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:23.305047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:23.305047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:23.305047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:23.305047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:23.305047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:23.305047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:23.305047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:23.305047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:23.305047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:23.305047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:23.305047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:23.305047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:23.305047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:23.306028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:23.306028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:23.306028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:23.306222 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 346/700] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:23.315868 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491a506-287b-41fd-9673-c0667e0b4e8a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:23.326321 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e5222-54fd-4ea1-afbe-aa1eed0e00fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:23.327460 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e5222-54fd-4ea1-afbe-aa1eed0e00fa None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:23.327952 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e5222-54fd-4ea1-afbe-aa1eed0e00fa None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:23.328492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e5222-54fd-4ea1-afbe-aa1eed0e00fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:23.341632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e5222-54fd-4ea1-afbe-aa1eed0e00fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:23.344243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e5222-54fd-4ea1-afbe-aa1eed0e00fa None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:23.361939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-4d3e5222-54fd-4ea1-afbe-aa1eed0e00fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:23.361939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:23.361939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:23.361939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:23.361939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:23.361939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:23.361939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:23.361939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:23.361939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:23.361939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:23.361939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:23.361939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:23.361939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:23.361939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:23.361939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:23.361939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:23.362984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:23.362984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:23.362984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:23.362984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:23.362984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:23.362984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:23.362984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:23.362984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:23.362984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:23.362984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:23.362984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:23.362984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:23.362984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:23.362984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:23.362984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:23.362984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:23.362984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:23.363958 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:23.363958 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:23.363958 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:23.367407 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 355/701] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:31:23.373391 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e5222-54fd-4ea1-afbe-aa1eed0e00fa None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:23.384328 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:23.487061 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:23.495061 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e5222-54fd-4ea1-afbe-aa1eed0e00fa None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:23.511305 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491a506-287b-41fd-9673-c0667e0b4e8a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:23.536999 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f098c-0806-43b1-9e34-ccc43e181ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:23.537924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f098c-0806-43b1-9e34-ccc43e181ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:23.538361 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f098c-0806-43b1-9e34-ccc43e181ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:23.538784 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f098c-0806-43b1-9e34-ccc43e181ac1 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:23.539595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f098c-0806-43b1-9e34-ccc43e181ac1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:23.540088 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f098c-0806-43b1-9e34-ccc43e181ac1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:23.541664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f098c-0806-43b1-9e34-ccc43e181ac1 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:23.555596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 356/702] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:31:23.561704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f098c-0806-43b1-9e34-ccc43e181ac1 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:23.575452 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059990f-80da-40d5-ac6a-6737a0bba817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:23.576325 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059990f-80da-40d5-ac6a-6737a0bba817 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:23.576794 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059990f-80da-40d5-ac6a-6737a0bba817 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:23.577150 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059990f-80da-40d5-ac6a-6737a0bba817 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:23.578135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059990f-80da-40d5-ac6a-6737a0bba817 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:23.578930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059990f-80da-40d5-ac6a-6737a0bba817 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:23.581025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059990f-80da-40d5-ac6a-6737a0bba817 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:23.609749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 347/703] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:23.617390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059990f-80da-40d5-ac6a-6737a0bba817 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:23.724189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f098c-0806-43b1-9e34-ccc43e181ac1 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:23.763715 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc682452-5bd7-426f-aa3d-36fd57ac412c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:23.764327 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc682452-5bd7-426f-aa3d-36fd57ac412c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:23.764600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc682452-5bd7-426f-aa3d-36fd57ac412c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:23.764871 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc682452-5bd7-426f-aa3d-36fd57ac412c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:23.772593 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc682452-5bd7-426f-aa3d-36fd57ac412c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:23.774182 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc682452-5bd7-426f-aa3d-36fd57ac412c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:23.778106 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:23.789723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-bc682452-5bd7-426f-aa3d-36fd57ac412c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:23.789723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:23.789723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:23.789723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:23.789723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:23.789723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:23.789723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:23.789723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:23.789723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:23.789723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:23.789723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:23.789723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:23.789723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:23.789723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:23.789723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:23.791648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:23.791648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:23.791648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:23.791648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:23.791648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:23.791648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:23.791648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:23.791648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:23.791648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:23.791648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:23.791648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:23.791648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:23.791648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:23.791648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:23.791648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:23.791648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:23.791648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:23.793472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:23.793472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:23.793472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:23.793472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:23.793472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:23.793472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:23.793472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:23.793472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:23.793472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 357/704] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:31:23.793472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f167b8c-c671-4a21-b9c7-3027ec543df2 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:23.793472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f167b8c-c671-4a21-b9c7-3027ec543df2 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:23.794667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f167b8c-c671-4a21-b9c7-3027ec543df2 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:23.798639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059990f-80da-40d5-ac6a-6737a0bba817 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:23.798853 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 358/705] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:23.799599 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f167b8c-c671-4a21-b9c7-3027ec543df2 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:23.817199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:23.858747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab1bc1-7f50-47d6-9a0a-ca5af8d5b96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:23.859876 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab1bc1-7f50-47d6-9a0a-ca5af8d5b96b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:23.860427 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab1bc1-7f50-47d6-9a0a-ca5af8d5b96b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:23.861028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab1bc1-7f50-47d6-9a0a-ca5af8d5b96b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:23.872456 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab1bc1-7f50-47d6-9a0a-ca5af8d5b96b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:23.875546 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab1bc1-7f50-47d6-9a0a-ca5af8d5b96b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:23.900837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-d7ab1bc1-7f50-47d6-9a0a-ca5af8d5b96b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:23.900837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:23.900837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:23.900837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:23.900837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:23.900837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:23.900837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:23.900837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:23.900837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:23.900837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:23.900837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:23.900837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:23.900837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:23.900837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:23.900837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:23.902674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:23.902674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:23.902674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:23.902674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:23.902674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:23.902674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:23.902674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:23.902674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:23.902674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:23.902674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:23.902674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:23.902674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:23.902674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:23.902674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:23.902674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:23.902674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:23.902674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:23.904520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:23.904520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:23.904520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:23.904520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:23.904520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:23.904520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:23.904520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:23.904520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:23.908976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 348/706] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:31:23.917450 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f167b8c-c671-4a21-b9c7-3027ec543df2 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:23.918320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531c871-01a8-44ef-8c94-a8d61548bb70 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:23.919060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531c871-01a8-44ef-8c94-a8d61548bb70 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:23.919748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531c871-01a8-44ef-8c94-a8d61548bb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:23.952422 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-6531c871-01a8-44ef-8c94-a8d61548bb70 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:23.954298 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6531c871-01a8-44ef-8c94-a8d61548bb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:23.973723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00196d-1f5d-4b8e-8465-3f8d0d9af908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:23.974744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00196d-1f5d-4b8e-8465-3f8d0d9af908 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:23.975241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00196d-1f5d-4b8e-8465-3f8d0d9af908 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:23.975728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00196d-1f5d-4b8e-8465-3f8d0d9af908 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:23.976724 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00196d-1f5d-4b8e-8465-3f8d0d9af908 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:23.977249 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00196d-1f5d-4b8e-8465-3f8d0d9af908 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:23.978979 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00196d-1f5d-4b8e-8465-3f8d0d9af908 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:23.993577 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 359/707] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:23.994925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00196d-1f5d-4b8e-8465-3f8d0d9af908 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:24.085768 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:24.095881 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 349/708] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:31:24.097987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531c871-01a8-44ef-8c94-a8d61548bb70 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:24.123352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00196d-1f5d-4b8e-8465-3f8d0d9af908 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:24.219667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b95b46-2fa7-490b-a1a6-5190a1754279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:24.221001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b95b46-2fa7-490b-a1a6-5190a1754279 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:24.222443 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b95b46-2fa7-490b-a1a6-5190a1754279 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:24.222968 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b95b46-2fa7-490b-a1a6-5190a1754279 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:24.224671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b95b46-2fa7-490b-a1a6-5190a1754279 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:24.225291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:24.226376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b95b46-2fa7-490b-a1a6-5190a1754279 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:24.230880 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b95b46-2fa7-490b-a1a6-5190a1754279 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:24.248994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531c871-01a8-44ef-8c94-a8d61548bb70 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:24.265525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 360/709] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:24.273829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b95b46-2fa7-490b-a1a6-5190a1754279 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:24.299545 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d8b8b-aaf8-435d-b37d-05545bd8b039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:24.300376 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d8b8b-aaf8-435d-b37d-05545bd8b039 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:24.300780 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d8b8b-aaf8-435d-b37d-05545bd8b039 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:24.301146 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d8b8b-aaf8-435d-b37d-05545bd8b039 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:24.302012 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d8b8b-aaf8-435d-b37d-05545bd8b039 None admin] RBAC: Adding query filter params (role.id=f0fd911d7c9246aeb3d67299473a9f42, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=d5d98f5adfaf410580481417e7ba1638) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:24.302420 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d8b8b-aaf8-435d-b37d-05545bd8b039 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:24.304432 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d8b8b-aaf8-435d-b37d-05545bd8b039 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:24.314077 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 350/710] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:24 2022] GET /identity/v3/role_assignments?user.id=d5d98f5adfaf410580481417e7ba1638&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=f0fd911d7c9246aeb3d67299473a9f42 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:24.422310 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b95b46-2fa7-490b-a1a6-5190a1754279 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:24.468266 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b6cdbe-7e25-4f8d-9f11-62ee48568477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:24.469002 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6cdbe-7e25-4f8d-9f11-62ee48568477 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:24.469289 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6cdbe-7e25-4f8d-9f11-62ee48568477 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:24.469533 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6cdbe-7e25-4f8d-9f11-62ee48568477 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:24.470408 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b6cdbe-7e25-4f8d-9f11-62ee48568477 None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=cc95bbdafb0c4f34ad0662bc2e3ba4ec) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:24.470724 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b6cdbe-7e25-4f8d-9f11-62ee48568477 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:24.472344 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b6cdbe-7e25-4f8d-9f11-62ee48568477 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:24.474841 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:24.485862 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 361/711] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:24 2022] GET /identity/v3/role_assignments?user.id=cc95bbdafb0c4f34ad0662bc2e3ba4ec&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:24.583440 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44996c5e-28e1-4fef-9385-915042a22d5f None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:24.583718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44996c5e-28e1-4fef-9385-915042a22d5f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:24.583987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44996c5e-28e1-4fef-9385-915042a22d5f None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:24.585609 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 351/712] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:24.596513 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9595ec-2409-4b25-9da0-9a0e838126af None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:24.597498 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9595ec-2409-4b25-9da0-9a0e838126af None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:24.598270 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9595ec-2409-4b25-9da0-9a0e838126af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:24.636260 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-4a9595ec-2409-4b25-9da0-9a0e838126af None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:24.636260 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4a9595ec-2409-4b25-9da0-9a0e838126af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:24.771384 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 362/713] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:31:24.775919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44996c5e-28e1-4fef-9385-915042a22d5f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:24.973609 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44996c5e-28e1-4fef-9385-915042a22d5f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:25.046757 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29552fc-6381-4ecb-adf2-7aaa343f7b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:25.048449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29552fc-6381-4ecb-adf2-7aaa343f7b23 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:25.049170 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29552fc-6381-4ecb-adf2-7aaa343f7b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:25.049908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29552fc-6381-4ecb-adf2-7aaa343f7b23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:25.072775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29552fc-6381-4ecb-adf2-7aaa343f7b23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:25.076259 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29552fc-6381-4ecb-adf2-7aaa343f7b23 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:25.099299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:25.109458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-d29552fc-6381-4ecb-adf2-7aaa343f7b23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:25.109458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:25.109458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:25.109458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:25.109458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:25.109458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:25.109458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:25.109458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:25.109458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:25.109458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:25.109458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:25.109458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:25.109458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:25.109458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:25.109458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:25.109458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:25.113009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:25.113009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:25.113009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:25.113009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:25.113009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:25.113009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:25.113009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:25.113009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:25.113009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:25.113009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:25.113009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:25.113009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:25.113009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:25.113009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:25.113009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:25.113009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:25.113009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:25.115390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:25.115390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:25.115390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:25.115390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 352/714] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:25.117677 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9595ec-2409-4b25-9da0-9a0e838126af None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:25.216480 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29552fc-6381-4ecb-adf2-7aaa343f7b23 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:25.363418 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9595ec-2409-4b25-9da0-9a0e838126af None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:25.423477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:25.449096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb8232-3813-4b7b-afe8-b8cfe9259131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:25.450667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb8232-3813-4b7b-afe8-b8cfe9259131 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:25.451419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb8232-3813-4b7b-afe8-b8cfe9259131 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:25.452314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb8232-3813-4b7b-afe8-b8cfe9259131 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:25.453917 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb8232-3813-4b7b-afe8-b8cfe9259131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:25.459056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb8232-3813-4b7b-afe8-b8cfe9259131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:25.462459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb8232-3813-4b7b-afe8-b8cfe9259131 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:25.488786 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 363/715] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:25.494363 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb8232-3813-4b7b-afe8-b8cfe9259131 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:25.526631 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29552fc-6381-4ecb-adf2-7aaa343f7b23 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:25.614211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ede69-f968-4984-9631-001a10a012a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:25.616863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ede69-f968-4984-9631-001a10a012a1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:25.617816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ede69-f968-4984-9631-001a10a012a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:25.618562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ede69-f968-4984-9631-001a10a012a1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:25.620114 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ede69-f968-4984-9631-001a10a012a1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:25.634829 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ede69-f968-4984-9631-001a10a012a1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:25.654562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-225ede69-f968-4984-9631-001a10a012a1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:31:25.654562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:25.654562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:31:25.654562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:31:25.654562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:31:25.654562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:31:25.654562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:31:25.654562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:25.654562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:31:25.654562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:25.654562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:25.654562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:25.654562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:25.654562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:25.654562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:25.654562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:25.654562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:25.656564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:25.656564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:25.656564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:25.656564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:25.656564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:25.656564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:25.656564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:31:25.656564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:31:25.656564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:31:25.656564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:31:25.656564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:25.656564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:25.656564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:25.656564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:25.656564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:25.656564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:25.659019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:25.659019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:25.659019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:25.659019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:25.659019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:25.659019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:25.659019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:25.659019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:25.659019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:25.659019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:25.659019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:31:25.659019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:25.659019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:31:25.659019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:25.664169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 353/716] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:25 2022] GET /identity/v3/services/network => generated 88 bytes in 448 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:31:25.675799 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ede69-f968-4984-9631-001a10a012a1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:25.729118 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb8232-3813-4b7b-afe8-b8cfe9259131 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:25.781929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027c350-333c-4870-b7c2-a1620a487df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:25.782573 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027c350-333c-4870-b7c2-a1620a487df3 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:25.782858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027c350-333c-4870-b7c2-a1620a487df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:25.783135 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027c350-333c-4870-b7c2-a1620a487df3 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:25.783813 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027c350-333c-4870-b7c2-a1620a487df3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:25.785517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027c350-333c-4870-b7c2-a1620a487df3 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:25.794530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:25.826904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 364/717] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:31:25 2022] POST /identity/v3/users => generated 254 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:31:25.885372 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ede69-f968-4984-9631-001a10a012a1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:25.969195 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0da6a7-5546-4b56-9257-3cb22f3629fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:25.970240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0da6a7-5546-4b56-9257-3cb22f3629fa None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:25.970724 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0da6a7-5546-4b56-9257-3cb22f3629fa None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:25.971203 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0da6a7-5546-4b56-9257-3cb22f3629fa None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:25.972396 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0da6a7-5546-4b56-9257-3cb22f3629fa None admin] RBAC: Adding query filter params (name=network) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:25.972953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0da6a7-5546-4b56-9257-3cb22f3629fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:25.975768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0da6a7-5546-4b56-9257-3cb22f3629fa None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:25.985400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 354/718] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Aug 30 14:31:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:25.991632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027c350-333c-4870-b7c2-a1620a487df3 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:26.061716 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:26.129156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027c350-333c-4870-b7c2-a1620a487df3 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:26.176322 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197edf73-4b08-415a-90ee-379f54745573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:26.177269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197edf73-4b08-415a-90ee-379f54745573 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:26.177753 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197edf73-4b08-415a-90ee-379f54745573 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:26.178226 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197edf73-4b08-415a-90ee-379f54745573 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:26.179390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197edf73-4b08-415a-90ee-379f54745573 None admin] RBAC: Adding query filter params (type=network) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:26.180041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197edf73-4b08-415a-90ee-379f54745573 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:26.182637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197edf73-4b08-415a-90ee-379f54745573 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:26.196620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 365/719] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Aug 30 14:31:25 2022] GET /identity/v3/services?type=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:26.236082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2568e-1519-43da-b7dd-5956900b83a1 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:26.236529 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2568e-1519-43da-b7dd-5956900b83a1 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:26.236978 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2568e-1519-43da-b7dd-5956900b83a1 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:26.239722 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 355/720] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:26.249696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9395d7-4fec-4c14-a848-3110e2c0e25c None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:26.250199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9395d7-4fec-4c14-a848-3110e2c0e25c None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:26.250755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9395d7-4fec-4c14-a848-3110e2c0e25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:26.289701 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-6a9395d7-4fec-4c14-a848-3110e2c0e25c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:26.292017 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6a9395d7-4fec-4c14-a848-3110e2c0e25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:26.428692 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 366/721] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:31:26.932853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2568e-1519-43da-b7dd-5956900b83a1 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:27.058473 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c7f26-0afd-4693-95a0-334bd10dc0b7 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:27.058772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c7f26-0afd-4693-95a0-334bd10dc0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:27.059111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c7f26-0afd-4693-95a0-334bd10dc0b7 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:27.060804 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 367/722] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:27.067638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ecd5f-3e26-484c-a739-de0fdd28021e None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:27.067912 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ecd5f-3e26-484c-a739-de0fdd28021e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:27.068254 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ecd5f-3e26-484c-a739-de0fdd28021e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:27.117876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-494ecd5f-3e26-484c-a739-de0fdd28021e None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:27.119969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-494ecd5f-3e26-484c-a739-de0fdd28021e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:27.131045 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2568e-1519-43da-b7dd-5956900b83a1 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:27.210258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:27.223173 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:27.234434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c280769-6315-4df8-a43e-63f43b3e0660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:27.235091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c280769-6315-4df8-a43e-63f43b3e0660 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:27.235486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c280769-6315-4df8-a43e-63f43b3e0660 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:27.235785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c280769-6315-4df8-a43e-63f43b3e0660 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:27.236487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c280769-6315-4df8-a43e-63f43b3e0660 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:27.238435 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c280769-6315-4df8-a43e-63f43b3e0660 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:27.280388 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 356/723] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Aug 30 14:31:26 2022] POST /identity/v3/services => generated 251 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:31:27.332511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 368/724] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:31:27.644392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c280769-6315-4df8-a43e-63f43b3e0660 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:27.782692 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01898aa3-20a3-4d78-834e-c0509a6af2b2 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:27.783525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01898aa3-20a3-4d78-834e-c0509a6af2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:27.783525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01898aa3-20a3-4d78-834e-c0509a6af2b2 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:27.784770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 369/725] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:27.792490 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c211c33-94f2-4901-a09d-4b9338ecf95f None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:27.792758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c211c33-94f2-4901-a09d-4b9338ecf95f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:27.793093 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c211c33-94f2-4901-a09d-4b9338ecf95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:27.820594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-5c211c33-94f2-4901-a09d-4b9338ecf95f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:27.821951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5c211c33-94f2-4901-a09d-4b9338ecf95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:27.854222 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c280769-6315-4df8-a43e-63f43b3e0660 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:27.918513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced58f2-762d-4c82-a1b3-1a601f7a6912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:27.919512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced58f2-762d-4c82-a1b3-1a601f7a6912 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:27.919991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced58f2-762d-4c82-a1b3-1a601f7a6912 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:27.920607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced58f2-762d-4c82-a1b3-1a601f7a6912 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:27.932294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 370/726] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:31:27.947072 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced58f2-762d-4c82-a1b3-1a601f7a6912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:27.955278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-bced58f2-762d-4c82-a1b3-1a601f7a6912 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:27.955278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:27.955278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:27.955278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:27.955278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:27.955278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:27.955278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:27.955278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:27.955278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:27.955278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:27.955278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:27.955278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:27.955278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:27.955278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:27.955278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:27.955278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:27.959181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:27.959181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:27.959181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:27.959181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:27.959181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:27.959181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:27.959181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:27.959181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:27.959181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:27.959181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:27.959181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:27.959181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:27.959181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:27.959181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:27.959181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:27.959181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:27.966893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:27.966893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:27.966893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:27.966893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:27.966893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:27.966893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:27.966893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:27.966893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:27.966893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:27.966893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:27.966893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:27.966893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:27.966893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 357/727] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:27.969653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c211c33-94f2-4901-a09d-4b9338ecf95f None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:28.143687 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:28.152693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c211c33-94f2-4901-a09d-4b9338ecf95f None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:28.205827 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4499e-fd0b-48ef-ac63-d4629097e36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:28.206436 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4499e-fd0b-48ef-ac63-d4629097e36e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:28.206693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4499e-fd0b-48ef-ac63-d4629097e36e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:28.207363 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4499e-fd0b-48ef-ac63-d4629097e36e None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:28.209377 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4499e-fd0b-48ef-ac63-d4629097e36e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:28.210526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4499e-fd0b-48ef-ac63-d4629097e36e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:28.212148 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4499e-fd0b-48ef-ac63-d4629097e36e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:28.226322 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 371/728] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:31:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:28.232169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced58f2-762d-4c82-a1b3-1a601f7a6912 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:28.296267 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4499e-fd0b-48ef-ac63-d4629097e36e None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:28.298194 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:28.365698 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced58f2-762d-4c82-a1b3-1a601f7a6912 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:28.406342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2df89-90fe-44e4-9890-b5fcdeeac6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:28.407013 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2df89-90fe-44e4-9890-b5fcdeeac6d7 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:28.407291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2df89-90fe-44e4-9890-b5fcdeeac6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:28.407559 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2df89-90fe-44e4-9890-b5fcdeeac6d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:28.423404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2df89-90fe-44e4-9890-b5fcdeeac6d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:28.424964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2df89-90fe-44e4-9890-b5fcdeeac6d7 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:28.442021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-33a2df89-90fe-44e4-9890-b5fcdeeac6d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:28.442021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:28.442021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:28.442021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:28.442021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:28.442021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:28.442021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:28.442021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:28.442021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:28.442021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:28.442021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:28.442021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:28.442021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:28.442021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:28.442021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:28.442021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:28.443066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:28.443066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:28.443066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:28.443066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:28.443066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:28.443066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:28.443066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:28.443066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:28.443066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:28.443066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:28.443066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:28.443066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:28.443066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:28.443066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:28.443066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:28.443066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:28.443066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:28.444370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:28.444370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:28.444370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:28.444370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 358/729] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:28.451034 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2df89-90fe-44e4-9890-b5fcdeeac6d7 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:28.536878 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4499e-fd0b-48ef-ac63-d4629097e36e None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:28.613722 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437373f-882b-4243-bf39-1bafe712b0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:28.614829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437373f-882b-4243-bf39-1bafe712b0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:28.615327 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437373f-882b-4243-bf39-1bafe712b0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:28.618073 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437373f-882b-4243-bf39-1bafe712b0a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:28.639984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b437373f-882b-4243-bf39-1bafe712b0a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:28.651845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-b437373f-882b-4243-bf39-1bafe712b0a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:28.651845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:28.651845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:28.651845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:28.651845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:28.651845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:28.651845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:28.651845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:28.651845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:28.651845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:28.651845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:28.651845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:28.651845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:28.651845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:28.651845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:28.651845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:28.653637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:28.653637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:28.653637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:28.653637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:28.653637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:28.653637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:28.653637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:28.653637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:28.653637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:28.653637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:28.653637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:28.653637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:28.653637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:28.653637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:28.653637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:28.653637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:28.655213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:28.655213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:28.655213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:28.655213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:28.655213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:28.655213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:28.655213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:28.655213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:28.655213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:28.655213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:28.655213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:28.655213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:28.655213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 372/730] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:28.664845 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:28.665491 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437373f-882b-4243-bf39-1bafe712b0a9 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:28.677583 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2df89-90fe-44e4-9890-b5fcdeeac6d7 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:28.701314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:28.730803 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec53de1-f8a6-4a39-8d40-10c74a47dbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:28.731434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec53de1-f8a6-4a39-8d40-10c74a47dbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:28.731689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec53de1-f8a6-4a39-8d40-10c74a47dbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:28.731966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec53de1-f8a6-4a39-8d40-10c74a47dbc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:28.732674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec53de1-f8a6-4a39-8d40-10c74a47dbc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:28.733194 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec53de1-f8a6-4a39-8d40-10c74a47dbc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:28.735404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec53de1-f8a6-4a39-8d40-10c74a47dbc0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:28.754436 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 359/731] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:31:28.756343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec53de1-f8a6-4a39-8d40-10c74a47dbc0 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:28.973302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec53de1-f8a6-4a39-8d40-10c74a47dbc0 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:28.974852 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437373f-882b-4243-bf39-1bafe712b0a9 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:29.033117 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afcbf5-912c-458d-924f-49957c08f387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:29.033778 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afcbf5-912c-458d-924f-49957c08f387 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:29.034054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afcbf5-912c-458d-924f-49957c08f387 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:29.034319 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afcbf5-912c-458d-924f-49957c08f387 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:29.038273 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782c1ef-1aef-4bd9-bbff-c73a39cb2873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:29.039394 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782c1ef-1aef-4bd9-bbff-c73a39cb2873 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:29.039883 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782c1ef-1aef-4bd9-bbff-c73a39cb2873 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:29.040389 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782c1ef-1aef-4bd9-bbff-c73a39cb2873 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:29.041543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782c1ef-1aef-4bd9-bbff-c73a39cb2873 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:29.042154 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782c1ef-1aef-4bd9-bbff-c73a39cb2873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:29.042646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afcbf5-912c-458d-924f-49957c08f387 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:29.044158 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afcbf5-912c-458d-924f-49957c08f387 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:29.044756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782c1ef-1aef-4bd9-bbff-c73a39cb2873 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:29.059265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-48afcbf5-912c-458d-924f-49957c08f387 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:31:29.059265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:29.059265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:29.059265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:29.059265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:29.059265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:29.059265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:29.059265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:29.059265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:29.059265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:29.059265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:29.059265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:29.059265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:29.059265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:29.059265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:29.059265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:29.061948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:29.064412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:29.064412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:29.064412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:29.064412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:29.064412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:29.064412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:29.064412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:29.064412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:29.064412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:31:29.064412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:29.064412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 360/733] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Aug 30 14:31:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:31:29.064412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545fcd8-4c7f-4475-8b3e-0917e36fc41c None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:29.064412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545fcd8-4c7f-4475-8b3e-0917e36fc41c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:29.066009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545fcd8-4c7f-4475-8b3e-0917e36fc41c None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:29.066167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 373/733] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:31:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:31:29.072060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 361/734] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:29.072388 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782c1ef-1aef-4bd9-bbff-c73a39cb2873 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:29.072757 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eeb868-3e88-4dca-a444-c63e53d378ac None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:29.073138 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eeb868-3e88-4dca-a444-c63e53d378ac None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:29.073503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eeb868-3e88-4dca-a444-c63e53d378ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:29.101448 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-49eeb868-3e88-4dca-a444-c63e53d378ac None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:29.103212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-49eeb868-3e88-4dca-a444-c63e53d378ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:29.122874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:29.200164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 362/735] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:31:29.202020 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eeb868-3e88-4dca-a444-c63e53d378ac None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:29.243384 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:29.314340 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782c1ef-1aef-4bd9-bbff-c73a39cb2873 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:29.445877 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abae1c9-7ab4-47df-bb40-e6e0283d7322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:29.447052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abae1c9-7ab4-47df-bb40-e6e0283d7322 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:29.447552 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abae1c9-7ab4-47df-bb40-e6e0283d7322 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:29.448052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abae1c9-7ab4-47df-bb40-e6e0283d7322 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:29.468456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abae1c9-7ab4-47df-bb40-e6e0283d7322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:29.471314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abae1c9-7ab4-47df-bb40-e6e0283d7322 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:29.499242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-8abae1c9-7ab4-47df-bb40-e6e0283d7322 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:29.499242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:29.499242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:29.499242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:29.499242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:29.499242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:29.499242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:29.499242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:29.499242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:29.499242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:29.499242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:29.499242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:29.499242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:29.499242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:29.499242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:29.499242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:29.500458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:29.500458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:29.500458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:29.500458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:29.500458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:29.500458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:29.500458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:29.500458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:29.500458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:29.500458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:29.500458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:29.500458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:29.500458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:29.500458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:29.500458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:29.500458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:29.500458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:29.501348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:29.501348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:29.501348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:29.501712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 374/736] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 432 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:29.509263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abae1c9-7ab4-47df-bb40-e6e0283d7322 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:29.525183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eeb868-3e88-4dca-a444-c63e53d378ac None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:29.595064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bd55f-30b0-4746-967b-ecc06a81ed10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:29.596404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bd55f-30b0-4746-967b-ecc06a81ed10 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:29.596404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bd55f-30b0-4746-967b-ecc06a81ed10 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:29.596695 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bd55f-30b0-4746-967b-ecc06a81ed10 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:29.597931 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3bd55f-30b0-4746-967b-ecc06a81ed10 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:29.598312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3bd55f-30b0-4746-967b-ecc06a81ed10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:29.600571 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3bd55f-30b0-4746-967b-ecc06a81ed10 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:29.634981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 363/737] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Aug 30 14:31:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:29.636532 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bd55f-30b0-4746-967b-ecc06a81ed10 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:29.657032 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:29.663320 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abae1c9-7ab4-47df-bb40-e6e0283d7322 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:29.709663 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:29.729393 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30112878-94c5-4bb8-ab65-9caf958a9295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:29.730440 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30112878-94c5-4bb8-ab65-9caf958a9295 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:29.731030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30112878-94c5-4bb8-ab65-9caf958a9295 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:29.731766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30112878-94c5-4bb8-ab65-9caf958a9295 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:29.733203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30112878-94c5-4bb8-ab65-9caf958a9295 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:29.733941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30112878-94c5-4bb8-ab65-9caf958a9295 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:29.736412 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30112878-94c5-4bb8-ab65-9caf958a9295 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:29.755014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 375/738] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:29.756189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30112878-94c5-4bb8-ab65-9caf958a9295 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:29.779262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bd55f-30b0-4746-967b-ecc06a81ed10 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:29.826171 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec866e5-5b4f-40b2-9298-652fd1b1416b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:29.827356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec866e5-5b4f-40b2-9298-652fd1b1416b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:29.827783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec866e5-5b4f-40b2-9298-652fd1b1416b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:29.828593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec866e5-5b4f-40b2-9298-652fd1b1416b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:29.843928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec866e5-5b4f-40b2-9298-652fd1b1416b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:29.850508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-4ec866e5-5b4f-40b2-9298-652fd1b1416b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:29.850508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:29.850508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:29.850508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:29.850508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:29.850508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:29.850508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:29.850508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:29.850508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:29.850508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:29.850508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:29.850508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:29.850508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:29.850508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:29.850508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:29.850508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:29.851632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:29.851632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:29.851632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:29.851632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:29.851632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:29.851632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:29.851632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:29.851632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:29.851632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:29.851632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:29.851632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:29.851632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:29.851632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:29.851632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:29.851632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:29.851632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:29.852701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:29.852701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:29.852701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:29.852701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:29.852701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:29.852701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:29.852701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:29.852701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:29.852701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:29.852701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:29.852701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:29.852701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:29.855075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 364/739] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:31:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:29.856100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec866e5-5b4f-40b2-9298-652fd1b1416b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:29.927629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30112878-94c5-4bb8-ab65-9caf958a9295 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:29.954283 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:29.998178 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7e25c-8fc8-45ba-9df5-dea51c80341e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:29.999149 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7e25c-8fc8-45ba-9df5-dea51c80341e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:29.999497 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7e25c-8fc8-45ba-9df5-dea51c80341e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:29.999923 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7e25c-8fc8-45ba-9df5-dea51c80341e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:30.004537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec866e5-5b4f-40b2-9298-652fd1b1416b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:30.019128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7e25c-8fc8-45ba-9df5-dea51c80341e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:30.021172 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7e25c-8fc8-45ba-9df5-dea51c80341e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:30.049641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-27b7e25c-8fc8-45ba-9df5-dea51c80341e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:30.049641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:30.049641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:30.049641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:30.049641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:30.049641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:30.049641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:30.049641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:30.049641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:30.049641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:30.049641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:30.049641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:30.049641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:30.049641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:30.049641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:30.049641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:30.051582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:30.051582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:30.051582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:30.051582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:30.051582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:30.051582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:30.051582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:30.051582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:30.051582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:30.051582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:30.051582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:30.051582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:30.051582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:30.051582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:30.051582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:30.051582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:30.051582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:30.053463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:30.053463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:30.053463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:30.055841 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 376/740] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:30.058187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e02727-1df7-436e-a41d-7a2292da2c31 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:30.058596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e02727-1df7-436e-a41d-7a2292da2c31 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:30.058998 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e02727-1df7-436e-a41d-7a2292da2c31 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:30.061695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 377/741] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:30.068554 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e02727-1df7-436e-a41d-7a2292da2c31 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:30.075630 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00491b4-1a1c-4daa-89a0-050bedc531b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:30.076340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00491b4-1a1c-4daa-89a0-050bedc531b4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:30.076703 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00491b4-1a1c-4daa-89a0-050bedc531b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:30.077033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00491b4-1a1c-4daa-89a0-050bedc531b4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:30.086541 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00491b4-1a1c-4daa-89a0-050bedc531b4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:30.088070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00491b4-1a1c-4daa-89a0-050bedc531b4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:30.102664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-d00491b4-1a1c-4daa-89a0-050bedc531b4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:31:30.102664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:30.102664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:30.102664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:30.102664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:30.102664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:30.102664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:30.102664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:30.102664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:30.102664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:30.102664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:30.102664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:30.102664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:30.102664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:30.102664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:30.102664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:30.104599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:30.106552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:30.106552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:30.106552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:30.106552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:30.106552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:30.106552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:30.106552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:30.106552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:30.106552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:31:30.106552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:30.106552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 365/742] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:30.106552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00491b4-1a1c-4daa-89a0-050bedc531b4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:30.146510 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:30.221951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e02727-1df7-436e-a41d-7a2292da2c31 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:30.247254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00491b4-1a1c-4daa-89a0-050bedc531b4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:30.260825 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e362b43-351c-4680-954b-f1086af97825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:30.261556 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e362b43-351c-4680-954b-f1086af97825 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:30.261885 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e362b43-351c-4680-954b-f1086af97825 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:30.262267 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e362b43-351c-4680-954b-f1086af97825 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:30.263124 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e362b43-351c-4680-954b-f1086af97825 None admin] RBAC: Adding query filter params (name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:30.263549 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e362b43-351c-4680-954b-f1086af97825 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:30.265241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e362b43-351c-4680-954b-f1086af97825 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:30.275437 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 378/743] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:31:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:30.276223 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:30.277290 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2d2fe-e398-457c-b462-a7a7de8ce17b None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:30.277464 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2d2fe-e398-457c-b462-a7a7de8ce17b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:30.277788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2d2fe-e398-457c-b462-a7a7de8ce17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:30.291633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491fcd1-f8d3-456d-b2b0-2fe042c6c0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:30.292286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491fcd1-f8d3-456d-b2b0-2fe042c6c0ab None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:30.292565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491fcd1-f8d3-456d-b2b0-2fe042c6c0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:30.292833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491fcd1-f8d3-456d-b2b0-2fe042c6c0ab None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:30.293513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e491fcd1-f8d3-456d-b2b0-2fe042c6c0ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:30.293813 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e491fcd1-f8d3-456d-b2b0-2fe042c6c0ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:30.295205 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e491fcd1-f8d3-456d-b2b0-2fe042c6c0ab None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:30.298088 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-c0a2d2fe-e398-457c-b462-a7a7de8ce17b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:30.299735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c0a2d2fe-e398-457c-b462-a7a7de8ce17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:30.316607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 366/744] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:30.317990 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491fcd1-f8d3-456d-b2b0-2fe042c6c0ab None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:30.392521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 379/745] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:31:30.394193 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e57ff-3ad6-4fae-9d7b-0f0c513a1f42 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:30.394576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e57ff-3ad6-4fae-9d7b-0f0c513a1f42 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:30.394969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e57ff-3ad6-4fae-9d7b-0f0c513a1f42 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:30.396558 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 380/746] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:30.397848 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e57ff-3ad6-4fae-9d7b-0f0c513a1f42 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:30.451160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:30.489065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491fcd1-f8d3-456d-b2b0-2fe042c6c0ab None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:30.553183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d50bf-a06f-4f4f-bd57-85d7255627c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:30.554329 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d50bf-a06f-4f4f-bd57-85d7255627c7 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:30.554757 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d50bf-a06f-4f4f-bd57-85d7255627c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:30.555201 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d50bf-a06f-4f4f-bd57-85d7255627c7 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:30.557250 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d50bf-a06f-4f4f-bd57-85d7255627c7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:30.557739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d50bf-a06f-4f4f-bd57-85d7255627c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:30.560289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d50bf-a06f-4f4f-bd57-85d7255627c7 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:30.561263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e57ff-3ad6-4fae-9d7b-0f0c513a1f42 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:30.600625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 367/747] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Aug 30 14:31:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:30.602509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d50bf-a06f-4f4f-bd57-85d7255627c7 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:30.615575 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf73a3b-f520-4770-ac61-ab0c4d011e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:30.616492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf73a3b-f520-4770-ac61-ab0c4d011e83 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:30.616904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf73a3b-f520-4770-ac61-ab0c4d011e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:30.617302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf73a3b-f520-4770-ac61-ab0c4d011e83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:30.636101 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf73a3b-f520-4770-ac61-ab0c4d011e83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:30.638023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf73a3b-f520-4770-ac61-ab0c4d011e83 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:30.657695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-edf73a3b-f520-4770-ac61-ab0c4d011e83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:30.657695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:30.657695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:30.657695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:30.657695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:30.657695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:30.657695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:30.657695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:30.657695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:30.657695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:30.657695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:30.657695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:30.657695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:30.657695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:30.657695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:30.657695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:30.659360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:30.659360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:30.659360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:30.659360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:30.659360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:30.659360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:30.659360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:30.659360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:30.659360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:30.659360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:30.659360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:30.659360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:30.659360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:30.659360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:30.659360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:30.659360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:30.659360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:30.660991 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:30.660991 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:30.660991 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:30.660991 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 381/748] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:30.662495 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec4f1b-6f95-4668-b58c-e2bd0d74f950 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:30.662973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec4f1b-6f95-4668-b58c-e2bd0d74f950 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:30.663987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec4f1b-6f95-4668-b58c-e2bd0d74f950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:30.689134 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-f1ec4f1b-6f95-4668-b58c-e2bd0d74f950 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:30.690367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f1ec4f1b-6f95-4668-b58c-e2bd0d74f950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:30.694818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:30.779166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 382/749] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:31:30.780713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec4f1b-6f95-4668-b58c-e2bd0d74f950 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:30.782903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d50bf-a06f-4f4f-bd57-85d7255627c7 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:30.815238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:30.830412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968c119-deec-4e71-870e-76dbf0cc8b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:30.831510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968c119-deec-4e71-870e-76dbf0cc8b1b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:30.832398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968c119-deec-4e71-870e-76dbf0cc8b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:30.832879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968c119-deec-4e71-870e-76dbf0cc8b1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:30.844273 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968c119-deec-4e71-870e-76dbf0cc8b1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:30.846575 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968c119-deec-4e71-870e-76dbf0cc8b1b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:30.863564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-8968c119-deec-4e71-870e-76dbf0cc8b1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:30.863564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:30.863564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:30.863564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:30.863564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:30.863564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:30.863564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:30.863564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:30.863564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:30.863564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:30.863564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:30.863564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:30.863564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:30.863564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:30.863564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:30.865641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:30.865641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:30.865641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:30.865641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:30.865641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:30.865641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:30.865641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:30.865641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:30.865641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:30.865641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:30.865641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:30.865641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:30.865641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:30.865641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:30.865641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:30.865641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:30.865641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:30.867485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:30.867485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:30.867485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:30.867485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:30.867485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:30.867485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:30.867485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:30.867485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:30.869797 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 368/750] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:30.870784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968c119-deec-4e71-870e-76dbf0cc8b1b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:30.978480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec4f1b-6f95-4668-b58c-e2bd0d74f950 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:30.989219 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968c119-deec-4e71-870e-76dbf0cc8b1b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:30.997584 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:31.031851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b18061-03e9-4daf-9ceb-079615d44a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:31.032523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b18061-03e9-4daf-9ceb-079615d44a1a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:31.032803 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b18061-03e9-4daf-9ceb-079615d44a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:31.033069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b18061-03e9-4daf-9ceb-079615d44a1a None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:31.033216 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa84d9-e94b-4e62-a6da-5a608fd44d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:31.033938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa84d9-e94b-4e62-a6da-5a608fd44d6c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:31.034070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b18061-03e9-4daf-9ceb-079615d44a1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:31.034199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa84d9-e94b-4e62-a6da-5a608fd44d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:31.034329 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b18061-03e9-4daf-9ceb-079615d44a1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:31.034466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa84d9-e94b-4e62-a6da-5a608fd44d6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:31.035401 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b18061-03e9-4daf-9ceb-079615d44a1a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:31.051234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 369/751] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:31.053074 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b18061-03e9-4daf-9ceb-079615d44a1a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:31.123386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa84d9-e94b-4e62-a6da-5a608fd44d6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:31.126816 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa84d9-e94b-4e62-a6da-5a608fd44d6c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:31.158720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:31.193312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b18061-03e9-4daf-9ceb-079615d44a1a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:31.232835 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac3529-13c4-4a24-8d2f-23da2faf38bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:31.233484 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac3529-13c4-4a24-8d2f-23da2faf38bb None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:31.233755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac3529-13c4-4a24-8d2f-23da2faf38bb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:31.234019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac3529-13c4-4a24-8d2f-23da2faf38bb None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:31.234680 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac3529-13c4-4a24-8d2f-23da2faf38bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:31.236071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac3529-13c4-4a24-8d2f-23da2faf38bb None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:31.256258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 370/752] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Aug 30 14:31:31 2022] POST /identity/v3/services => generated 240 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:31:31.256258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac3529-13c4-4a24-8d2f-23da2faf38bb None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:31.291609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 383/753] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:30 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:31:31.293811 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa84d9-e94b-4e62-a6da-5a608fd44d6c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:31.299654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:31.441455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac3529-13c4-4a24-8d2f-23da2faf38bb None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:31.442466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa84d9-e94b-4e62-a6da-5a608fd44d6c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:31.478946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:31.488164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199323c-eaee-4168-a24b-c3f67b0f0a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:31.488832 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199323c-eaee-4168-a24b-c3f67b0f0a40 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:31.489105 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199323c-eaee-4168-a24b-c3f67b0f0a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:31.489370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199323c-eaee-4168-a24b-c3f67b0f0a40 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:31.490087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199323c-eaee-4168-a24b-c3f67b0f0a40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:31.490525 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199323c-eaee-4168-a24b-c3f67b0f0a40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:31.491839 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199323c-eaee-4168-a24b-c3f67b0f0a40 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:31.501756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d3c10-5278-4f5e-ae50-9f0de1a385b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:31.502408 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d3c10-5278-4f5e-ae50-9f0de1a385b2 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:31.502687 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d3c10-5278-4f5e-ae50-9f0de1a385b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:31.502949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d3c10-5278-4f5e-ae50-9f0de1a385b2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:31.506788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 371/754] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:31.507686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199323c-eaee-4168-a24b-c3f67b0f0a40 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:31.517058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d3c10-5278-4f5e-ae50-9f0de1a385b2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:31.518521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d3c10-5278-4f5e-ae50-9f0de1a385b2 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:31.532315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-9f6d3c10-5278-4f5e-ae50-9f0de1a385b2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:31:31.532315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:31.532315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:31.532315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:31.532315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:31.532315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:31.532315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:31.532315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:31.532315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:31.532315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:31.532315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:31.532315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:31.532315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:31.532315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:31.532315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:31.532315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:31.534198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:31.536049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:31.536049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:31.536049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:31.536049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:31.536049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:31.536049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:31.536049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:31.536049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:31.536049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:31:31.536049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:31.536049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 384/755] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Aug 30 14:31:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:31.536049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d3c10-5278-4f5e-ae50-9f0de1a385b2 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:31.620280 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:31.647406 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199323c-eaee-4168-a24b-c3f67b0f0a40 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:31.686412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3bc1c0-641a-4073-90aa-14183c777e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:31.687141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bc1c0-641a-4073-90aa-14183c777e31 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:31.687416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bc1c0-641a-4073-90aa-14183c777e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:31.687687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bc1c0-641a-4073-90aa-14183c777e31 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:31.688420 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3bc1c0-641a-4073-90aa-14183c777e31 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:31.689820 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3bc1c0-641a-4073-90aa-14183c777e31 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:31.698260 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d3c10-5278-4f5e-ae50-9f0de1a385b2 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:31.699753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-7b3bc1c0-641a-4073-90aa-14183c777e31 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:31:31.699753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:31.699753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:31:31.699753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:31:31.699753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:31:31.699753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:31:31.699753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:31:31.699753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:31.699753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:31:31.699753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:31.699753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:31.699753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:31.699753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:31.699753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:31.699753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:31.699753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:31.699753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:31.701148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:31.701148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:31.701148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:31.701148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:31.701148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:31.701148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:31.701148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:31:31.701148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:31:31.701148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:31:31.701148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:31:31.701148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:31.701148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:31.701148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:31.701148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:31.701148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:31.701148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:31.702183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:31.702183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:31.702183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:31.702183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:31.702183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:31.702183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:31.702183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:31.702183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:31.702183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:31.702183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:31.702183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:31:31.702183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:31.702183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:31:31.702183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:31.702183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 372/756] 10.222.0.25 () {56 vars in 1129 bytes} [Tue Aug 30 14:31:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:31.703418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3bc1c0-641a-4073-90aa-14183c777e31 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:31.754449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1930682-9071-445c-a7ea-866d479f6e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:31.755121 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1930682-9071-445c-a7ea-866d479f6e3d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:31.755400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1930682-9071-445c-a7ea-866d479f6e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:31.755678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1930682-9071-445c-a7ea-866d479f6e3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:31.772109 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1930682-9071-445c-a7ea-866d479f6e3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:31.774576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1930682-9071-445c-a7ea-866d479f6e3d None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:31.794867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-d1930682-9071-445c-a7ea-866d479f6e3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:31.794867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:31.794867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:31.794867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:31.794867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:31.794867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:31.794867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:31.794867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:31.794867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:31.794867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:31.794867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:31.794867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:31.794867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:31.794867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:31.794867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:31.794867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:31.797203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:31.797203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:31.797203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:31.797203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:31.797203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:31.797203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:31.797203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:31.797203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:31.797203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:31.797203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:31.797203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:31.797203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:31.797203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:31.797203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:31.797203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:31.797203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:31.797203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:31.798986 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:31.798986 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:31.798986 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:31.798986 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 385/757] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:31.798986 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1930682-9071-445c-a7ea-866d479f6e3d None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:31.840621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:31.860114 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3bc1c0-641a-4073-90aa-14183c777e31 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:31.913257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960140f-ad42-40d1-be21-bcf7bc2d4a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:31.914347 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960140f-ad42-40d1-be21-bcf7bc2d4a97 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:31.914831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960140f-ad42-40d1-be21-bcf7bc2d4a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:31.915308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960140f-ad42-40d1-be21-bcf7bc2d4a97 None admin] PATH_INFO: `/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/cc95bbdafb0c4f34ad0662bc2e3ba4ec/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:31.918636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960140f-ad42-40d1-be21-bcf7bc2d4a97 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9c57a581324217ad2e64a0c9c7fdee, user_id=cc95bbdafb0c4f34ad0662bc2e3ba4ec, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:31.921423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960140f-ad42-40d1-be21-bcf7bc2d4a97 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:31.929171 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1930682-9071-445c-a7ea-866d479f6e3d None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:31.943243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 373/758] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:31:31 2022] PUT /identity/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/cc95bbdafb0c4f34ad0662bc2e3ba4ec/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:31:31.944913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960140f-ad42-40d1-be21-bcf7bc2d4a97 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:31.965342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:31.990745 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ca57f-7b3d-4d70-a50c-2f689cea97a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:31.991594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ca57f-7b3d-4d70-a50c-2f689cea97a9 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:31.991984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ca57f-7b3d-4d70-a50c-2f689cea97a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:31.992358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ca57f-7b3d-4d70-a50c-2f689cea97a9 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:31.993206 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567ca57f-7b3d-4d70-a50c-2f689cea97a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:31.993592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567ca57f-7b3d-4d70-a50c-2f689cea97a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:31.995372 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567ca57f-7b3d-4d70-a50c-2f689cea97a9 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:32.025002 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 386/759] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Aug 30 14:31:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:32.025701 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ca57f-7b3d-4d70-a50c-2f689cea97a9 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:32.144749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960140f-ad42-40d1-be21-bcf7bc2d4a97 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:32.197059 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f742de-5e2b-4de0-95cb-754ae70708ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:32.197747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f742de-5e2b-4de0-95cb-754ae70708ec None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:32.198031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f742de-5e2b-4de0-95cb-754ae70708ec None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:32.198304 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f742de-5e2b-4de0-95cb-754ae70708ec None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:32.198967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f742de-5e2b-4de0-95cb-754ae70708ec None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:32.199266 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f742de-5e2b-4de0-95cb-754ae70708ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:32.200861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f742de-5e2b-4de0-95cb-754ae70708ec None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:32.201377 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:32.207725 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 374/760] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Aug 30 14:31:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:32.209317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f742de-5e2b-4de0-95cb-754ae70708ec None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:32.230579 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ca57f-7b3d-4d70-a50c-2f689cea97a9 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:32.293582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acc08f-1b1a-4de4-86a7-fd117a4aca97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:32.294574 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acc08f-1b1a-4de4-86a7-fd117a4aca97 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:32.295030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acc08f-1b1a-4de4-86a7-fd117a4aca97 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:32.295335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:32.295624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acc08f-1b1a-4de4-86a7-fd117a4aca97 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:32.296753 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52acc08f-1b1a-4de4-86a7-fd117a4aca97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:32.297274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52acc08f-1b1a-4de4-86a7-fd117a4aca97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:32.299820 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52acc08f-1b1a-4de4-86a7-fd117a4aca97 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:32.322603 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 387/761] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:32.324131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acc08f-1b1a-4de4-86a7-fd117a4aca97 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:32.337641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f742de-5e2b-4de0-95cb-754ae70708ec None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:32.376390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c698d0d-b73b-4a8a-a0eb-f2d8da9a9185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:32.377130 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c698d0d-b73b-4a8a-a0eb-f2d8da9a9185 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:32.377397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c698d0d-b73b-4a8a-a0eb-f2d8da9a9185 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:32.377672 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c698d0d-b73b-4a8a-a0eb-f2d8da9a9185 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:32.461561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c698d0d-b73b-4a8a-a0eb-f2d8da9a9185 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:32.463064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c698d0d-b73b-4a8a-a0eb-f2d8da9a9185 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:32.515102 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acc08f-1b1a-4de4-86a7-fd117a4aca97 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:32.546005 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:32.593577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 375/762] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:32 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:31:32.594088 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c698d0d-b73b-4a8a-a0eb-f2d8da9a9185 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:32.596059 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c451fc-a177-4b6f-bdd2-e7264a981a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:32.597077 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c451fc-a177-4b6f-bdd2-e7264a981a51 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:32.597700 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c451fc-a177-4b6f-bdd2-e7264a981a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:32.597987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c451fc-a177-4b6f-bdd2-e7264a981a51 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:32.599903 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c451fc-a177-4b6f-bdd2-e7264a981a51 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:32.599903 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c451fc-a177-4b6f-bdd2-e7264a981a51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:32.602304 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c451fc-a177-4b6f-bdd2-e7264a981a51 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:32.617221 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 388/763] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Aug 30 14:31:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:32.617221 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c451fc-a177-4b6f-bdd2-e7264a981a51 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:32.651386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:32.733171 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c698d0d-b73b-4a8a-a0eb-f2d8da9a9185 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:32.774565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda638fc-9c30-4722-9423-1d5877e7ec3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:32.775620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda638fc-9c30-4722-9423-1d5877e7ec3b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:32.776180 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda638fc-9c30-4722-9423-1d5877e7ec3b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:32.776667 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda638fc-9c30-4722-9423-1d5877e7ec3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:32.785977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda638fc-9c30-4722-9423-1d5877e7ec3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:32.788115 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda638fc-9c30-4722-9423-1d5877e7ec3b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:32.802309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-dda638fc-9c30-4722-9423-1d5877e7ec3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:32.802309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:32.802309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:32.802309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:32.802309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:32.802309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:32.802309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:32.802309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:32.802309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:32.802309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:32.802309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:32.802309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:32.802309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:32.802309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:32.802309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:32.803291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:32.803291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:32.803291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:32.803291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:32.803291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:32.803291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:32.803291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:32.803291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:32.803291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:32.803291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:32.803291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:32.803291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:32.803291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:32.803291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:32.803291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:32.803291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:32.803291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:32.804234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:32.804234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:32.804234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:32.804234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:32.804234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:32.804234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:32.804234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:32.804234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:32.811884 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 376/764] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:32.813286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda638fc-9c30-4722-9423-1d5877e7ec3b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:32.826191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c451fc-a177-4b6f-bdd2-e7264a981a51 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:32.853507 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:32.894690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e753270-4ee8-4d95-84c5-7be10949c91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:32.896029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e753270-4ee8-4d95-84c5-7be10949c91f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:32.896249 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e753270-4ee8-4d95-84c5-7be10949c91f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:32.896720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e753270-4ee8-4d95-84c5-7be10949c91f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:32.914119 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e753270-4ee8-4d95-84c5-7be10949c91f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:32.917564 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e753270-4ee8-4d95-84c5-7be10949c91f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:32.934117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-1e753270-4ee8-4d95-84c5-7be10949c91f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:32.934117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:32.934117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:32.934117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:32.934117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:32.934117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:32.934117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:32.934117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:32.934117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:32.934117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:32.934117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:32.934117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:32.934117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:32.934117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:32.934117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:32.934117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:32.935175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:32.935175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:32.935175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:32.935175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:32.935175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:32.935175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:32.935175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:32.935175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:32.935175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:32.935175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:32.935175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:32.935175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:32.935175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:32.935175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:32.935175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:32.935175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:32.935175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:32.936241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:32.936241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:32.936241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:32.936241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 389/765] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:32.936241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e753270-4ee8-4d95-84c5-7be10949c91f None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:32.952236 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda638fc-9c30-4722-9423-1d5877e7ec3b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:33.002332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f2818-3d38-4312-a9ca-a21f7c753e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:33.003378 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f2818-3d38-4312-a9ca-a21f7c753e52 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:33.004062 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f2818-3d38-4312-a9ca-a21f7c753e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:33.004696 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f2818-3d38-4312-a9ca-a21f7c753e52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:33.006287 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f2818-3d38-4312-a9ca-a21f7c753e52 None admin] RBAC: Adding query filter params (interface=public, service_id=79906333b3444d7dbc0718c885e9f277, region_id=RegionOne) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:33.006902 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f2818-3d38-4312-a9ca-a21f7c753e52 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:33.008388 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:33.010671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f2818-3d38-4312-a9ca-a21f7c753e52 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:33.026109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 377/766] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Aug 30 14:31:32 2022] GET /identity/v3/endpoints?service_id=79906333b3444d7dbc0718c885e9f277&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:33.027318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f2818-3d38-4312-a9ca-a21f7c753e52 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:33.053037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e753270-4ee8-4d95-84c5-7be10949c91f None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:33.090566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3258a-7313-4889-8a05-609a90476ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:33.091516 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3258a-7313-4889-8a05-609a90476ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:33.091726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3258a-7313-4889-8a05-609a90476ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:33.092164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3258a-7313-4889-8a05-609a90476ea8 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:33.092981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3258a-7313-4889-8a05-609a90476ea8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:33.093515 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3258a-7313-4889-8a05-609a90476ea8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:33.095065 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3258a-7313-4889-8a05-609a90476ea8 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:33.113583 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 390/767] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:33.113583 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3258a-7313-4889-8a05-609a90476ea8 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:33.145252 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:33.154405 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f2818-3d38-4312-a9ca-a21f7c753e52 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:33.194798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f42361-9758-4266-a40d-181ec4722183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:33.195689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f42361-9758-4266-a40d-181ec4722183 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:33.196181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f42361-9758-4266-a40d-181ec4722183 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:33.196713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f42361-9758-4266-a40d-181ec4722183 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:33.197614 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f42361-9758-4266-a40d-181ec4722183 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:33.198103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f42361-9758-4266-a40d-181ec4722183 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:33.199799 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f42361-9758-4266-a40d-181ec4722183 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:33.219474 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 378/768] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:33.220808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f42361-9758-4266-a40d-181ec4722183 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:33.221733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3258a-7313-4889-8a05-609a90476ea8 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:33.247243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:33.261216 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88160adc-8439-4e8b-88d1-6f2127bf4d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:33.262001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88160adc-8439-4e8b-88d1-6f2127bf4d6c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:33.262364 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88160adc-8439-4e8b-88d1-6f2127bf4d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:33.262818 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88160adc-8439-4e8b-88d1-6f2127bf4d6c None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:33.263888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88160adc-8439-4e8b-88d1-6f2127bf4d6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:33.265760 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88160adc-8439-4e8b-88d1-6f2127bf4d6c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:33.277722 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 391/769] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Aug 30 14:31:33 2022] GET /identity/v3/services => generated 772 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:33.277722 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88160adc-8439-4e8b-88d1-6f2127bf4d6c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:33.382090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f42361-9758-4266-a40d-181ec4722183 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:33.432843 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f8c45-f0a0-4262-9d75-321718164a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:33.433499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f8c45-f0a0-4262-9d75-321718164a8a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:33.434055 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f8c45-f0a0-4262-9d75-321718164a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:33.434324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f8c45-f0a0-4262-9d75-321718164a8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:33.435100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f8c45-f0a0-4262-9d75-321718164a8a None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=d5d98f5adfaf410580481417e7ba1638) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:33.435442 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f8c45-f0a0-4262-9d75-321718164a8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:33.437048 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f8c45-f0a0-4262-9d75-321718164a8a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:33.443967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 379/770] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:33 2022] GET /identity/v3/role_assignments?user.id=d5d98f5adfaf410580481417e7ba1638&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:33.445335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4f206-fec4-411d-9477-fef5de1e595c None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:33.445622 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4f206-fec4-411d-9477-fef5de1e595c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:33.445881 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4f206-fec4-411d-9477-fef5de1e595c None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:33.452480 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 380/771] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:33.453950 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685e080-a961-4818-988e-99cfb67c3cd4 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:33.454234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685e080-a961-4818-988e-99cfb67c3cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:33.454511 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685e080-a961-4818-988e-99cfb67c3cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:33.475874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88160adc-8439-4e8b-88d1-6f2127bf4d6c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:33.478342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-5685e080-a961-4818-988e-99cfb67c3cd4 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:33.481285 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5685e080-a961-4818-988e-99cfb67c3cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:33.504446 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:33.541482 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f51c8-3cb8-4d12-b504-d5a8df4b3b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:33.542703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f51c8-3cb8-4d12-b504-d5a8df4b3b76 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:33.543182 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f51c8-3cb8-4d12-b504-d5a8df4b3b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:33.543697 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f51c8-3cb8-4d12-b504-d5a8df4b3b76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:33.558662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f51c8-3cb8-4d12-b504-d5a8df4b3b76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:33.561718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f51c8-3cb8-4d12-b504-d5a8df4b3b76 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:33.596726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-747f51c8-3cb8-4d12-b504-d5a8df4b3b76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:33.596726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:33.596726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:33.596726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:33.596726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:33.596726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:33.596726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:33.596726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:33.596726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:33.596726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:33.596726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:33.596726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:33.596726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:33.596726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:33.596726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:33.597737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:33.597737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:33.597737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:33.597737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:33.597737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:33.597737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:33.597737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:33.597737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:33.597737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:33.597737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:33.597737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:33.597737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:33.597737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:33.597737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:33.597737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:33.597737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:33.597737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:33.598649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:33.598649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:33.598649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:33.598649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:33.598649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:33.598649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:33.598649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:33.598649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:33.599552 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 392/772] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:33.607884 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f51c8-3cb8-4d12-b504-d5a8df4b3b76 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:33.614410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 381/773] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:33.680622 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:33.738698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f51c8-3cb8-4d12-b504-d5a8df4b3b76 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:33.780232 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17df4d-e939-462e-92f6-ee409729c6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:33.781264 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17df4d-e939-462e-92f6-ee409729c6ac None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:33.781754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17df4d-e939-462e-92f6-ee409729c6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:33.782216 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17df4d-e939-462e-92f6-ee409729c6ac None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:33.783132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17df4d-e939-462e-92f6-ee409729c6ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:33.783627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17df4d-e939-462e-92f6-ee409729c6ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:33.789402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17df4d-e939-462e-92f6-ee409729c6ac None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:33.803349 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 393/774] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:33.813187 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685e080-a961-4818-988e-99cfb67c3cd4 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:33.968974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17df4d-e939-462e-92f6-ee409729c6ac None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:33.975556 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685e080-a961-4818-988e-99cfb67c3cd4 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:34.026547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b77a4d-67d6-4308-a51c-e37194ad89d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:34.027187 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b77a4d-67d6-4308-a51c-e37194ad89d4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:34.027461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b77a4d-67d6-4308-a51c-e37194ad89d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:34.027757 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b77a4d-67d6-4308-a51c-e37194ad89d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:34.028748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b77a4d-67d6-4308-a51c-e37194ad89d4 None admin] RBAC: Adding query filter params (role.id=f0fd911d7c9246aeb3d67299473a9f42, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=8fc940594a1848aa9644b645049a1bd6) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:34.029165 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b77a4d-67d6-4308-a51c-e37194ad89d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:34.030767 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b77a4d-67d6-4308-a51c-e37194ad89d4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:34.051654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 382/775] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:33 2022] GET /identity/v3/role_assignments?user.id=8fc940594a1848aa9644b645049a1bd6&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=f0fd911d7c9246aeb3d67299473a9f42 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:34.191965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17df4d-e939-462e-92f6-ee409729c6ac None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:34.206484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:34.270646 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fe33a-7e1e-46c1-9fae-14ec90ec29e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:34.271622 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fe33a-7e1e-46c1-9fae-14ec90ec29e1 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:34.271622 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fe33a-7e1e-46c1-9fae-14ec90ec29e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:34.272130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fe33a-7e1e-46c1-9fae-14ec90ec29e1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:34.273137 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1fe33a-7e1e-46c1-9fae-14ec90ec29e1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:34.274859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1fe33a-7e1e-46c1-9fae-14ec90ec29e1 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:34.296367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-3d1fe33a-7e1e-46c1-9fae-14ec90ec29e1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:31:34.296367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:34.296367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:31:34.296367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:31:34.296367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:31:34.296367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:31:34.296367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:31:34.296367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:34.296367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:31:34.296367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:34.296367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:34.296367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:34.296367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:34.296367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:34.296367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:34.296367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:34.296367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:34.297682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:34.297682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:34.297682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:34.297682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:34.297682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:34.297682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:34.297682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:31:34.297682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:31:34.297682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:31:34.297682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:31:34.297682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:34.297682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:34.297682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:34.297682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:34.297682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:34.297682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:34.298886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:34.298886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:34.298886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:34.298886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:34.298886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:34.298886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:34.298886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:34.298886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:34.298886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:34.298886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:34.298886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:31:34.298886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:34.298886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:31:34.298886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:34.298886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 394/776] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:33 2022] GET /identity/v3/services/network => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:34.307322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b77a4d-67d6-4308-a51c-e37194ad89d4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:34.337837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:34.487226 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b77a4d-67d6-4308-a51c-e37194ad89d4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:34.531655 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d257d4-25f5-4d86-8c99-16636711d918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:34.532342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d257d4-25f5-4d86-8c99-16636711d918 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:34.532619 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d257d4-25f5-4d86-8c99-16636711d918 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:34.532912 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d257d4-25f5-4d86-8c99-16636711d918 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:34.533555 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d257d4-25f5-4d86-8c99-16636711d918 None admin] RBAC: Adding query filter params (name=network) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:34.533851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d257d4-25f5-4d86-8c99-16636711d918 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:34.535242 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d257d4-25f5-4d86-8c99-16636711d918 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:34.540436 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 383/777] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Aug 30 14:31:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:34.549507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fe33a-7e1e-46c1-9fae-14ec90ec29e1 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:34.621595 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db19e39-3bd5-4af4-a33e-8da1a7d5b39d None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:34.621860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db19e39-3bd5-4af4-a33e-8da1a7d5b39d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:34.622155 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db19e39-3bd5-4af4-a33e-8da1a7d5b39d None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:34.623920 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 384/778] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:31:34.637897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca2bb0-b909-4677-b6cf-c197ff204319 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:34.638265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca2bb0-b909-4677-b6cf-c197ff204319 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:34.638443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca2bb0-b909-4677-b6cf-c197ff204319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:34.665289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-62ca2bb0-b909-4677-b6cf-c197ff204319 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:34.666506 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-62ca2bb0-b909-4677-b6cf-c197ff204319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:34.720328 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fe33a-7e1e-46c1-9fae-14ec90ec29e1 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:34.792588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a64677-3e5d-4dab-a206-9e2cd795b734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:34.793115 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a64677-3e5d-4dab-a206-9e2cd795b734 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:34.793396 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a64677-3e5d-4dab-a206-9e2cd795b734 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:34.793664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a64677-3e5d-4dab-a206-9e2cd795b734 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:34.794295 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a64677-3e5d-4dab-a206-9e2cd795b734 None admin] RBAC: Adding query filter params (type=network) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:34.794593 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a64677-3e5d-4dab-a206-9e2cd795b734 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:34.796338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a64677-3e5d-4dab-a206-9e2cd795b734 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:34.808314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 395/779] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Aug 30 14:31:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:34.815862 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a64677-3e5d-4dab-a206-9e2cd795b734 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:34.823728 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 385/780] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:34.876880 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:34.974360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a64677-3e5d-4dab-a206-9e2cd795b734 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:35.026695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652c5fb-a514-42d6-a1a1-f364e9431a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:35.027330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652c5fb-a514-42d6-a1a1-f364e9431a61 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:35.027621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652c5fb-a514-42d6-a1a1-f364e9431a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:35.027883 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652c5fb-a514-42d6-a1a1-f364e9431a61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:35.028639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652c5fb-a514-42d6-a1a1-f364e9431a61 None admin] RBAC: Adding query filter params (interface=public, service_id=1ed961287fb745f89de274462c2947b7, region_id=RegionOne) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:35.028946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652c5fb-a514-42d6-a1a1-f364e9431a61 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:35.030637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652c5fb-a514-42d6-a1a1-f364e9431a61 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:35.042787 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 396/781] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Aug 30 14:31:34 2022] GET /identity/v3/endpoints?service_id=1ed961287fb745f89de274462c2947b7&interface=public®ion_id=RegionOne => generated 194 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:35.051280 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca2bb0-b909-4677-b6cf-c197ff204319 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:35.082801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:35.169000 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652c5fb-a514-42d6-a1a1-f364e9431a61 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:35.303934 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca2bb0-b909-4677-b6cf-c197ff204319 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:35.308371 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:35.312974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652c5fb-a514-42d6-a1a1-f364e9431a61 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:35.349624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab5948-94a4-4db7-9041-eb0d3d33db1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:35.350263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab5948-94a4-4db7-9041-eb0d3d33db1d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:35.350538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab5948-94a4-4db7-9041-eb0d3d33db1d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:35.350805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab5948-94a4-4db7-9041-eb0d3d33db1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:35.362342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab5948-94a4-4db7-9041-eb0d3d33db1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:35.367725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-1bab5948-94a4-4db7-9041-eb0d3d33db1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:35.367725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:35.367725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:35.367725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:35.367725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:35.367725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:35.367725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:35.367725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:35.367725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:35.367725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:35.367725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:35.367725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:35.367725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:35.367725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:35.367725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:35.367725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:35.368781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:35.368781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:35.368781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:35.368781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:35.368781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:35.368781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:35.368781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:35.368781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:35.368781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:35.368781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:35.368781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:35.368781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:35.368781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:35.368781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:35.368781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:35.368781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:35.369705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:35.369705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:35.369705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:35.369705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:35.369705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:35.369705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:35.369705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:35.369705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:35.369705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:35.369705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:35.369705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:35.369705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:35.369705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 397/782] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:31:35.376751 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab5948-94a4-4db7-9041-eb0d3d33db1d None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:35.398142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22d4d2-50c4-4b58-8cab-036a51f04660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:35.398777 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22d4d2-50c4-4b58-8cab-036a51f04660 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:35.399139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22d4d2-50c4-4b58-8cab-036a51f04660 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:35.399444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22d4d2-50c4-4b58-8cab-036a51f04660 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:35.400100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22d4d2-50c4-4b58-8cab-036a51f04660 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:35.401540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22d4d2-50c4-4b58-8cab-036a51f04660 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:35.415825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 386/783] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Aug 30 14:31:35 2022] GET /identity/v3/services => generated 772 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:35.490136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab5948-94a4-4db7-9041-eb0d3d33db1d None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:35.531278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698b54c-bd3c-4ccc-a932-73d91ac7b52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:35.531967 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698b54c-bd3c-4ccc-a932-73d91ac7b52a None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:35.532276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698b54c-bd3c-4ccc-a932-73d91ac7b52a None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:35.532542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698b54c-bd3c-4ccc-a932-73d91ac7b52a None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:35.533230 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698b54c-bd3c-4ccc-a932-73d91ac7b52a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:35.533528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698b54c-bd3c-4ccc-a932-73d91ac7b52a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:35.534966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698b54c-bd3c-4ccc-a932-73d91ac7b52a None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:35.544727 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 398/784] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:31:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:31:35.553598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22d4d2-50c4-4b58-8cab-036a51f04660 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:35.773747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:35.799438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22d4d2-50c4-4b58-8cab-036a51f04660 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:35.840229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404e931-44c6-42d5-a43f-0c816885ea77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:35.840904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404e931-44c6-42d5-a43f-0c816885ea77 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:35.842066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404e931-44c6-42d5-a43f-0c816885ea77 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:35.842066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404e931-44c6-42d5-a43f-0c816885ea77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:35.864352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404e931-44c6-42d5-a43f-0c816885ea77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:35.865933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404e931-44c6-42d5-a43f-0c816885ea77 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:35.889913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-0404e931-44c6-42d5-a43f-0c816885ea77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:35.889913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:35.889913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:35.889913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:35.889913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:35.889913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:35.889913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:35.889913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:35.889913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:35.889913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:35.889913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:35.889913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:35.889913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:35.889913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:35.889913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:35.889913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:35.890930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:35.890930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:35.890930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:35.890930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:35.890930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:35.890930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:35.890930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:35.890930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:35.890930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:35.890930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:35.890930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:35.890930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:35.890930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:35.890930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:35.890930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:35.890930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:35.890930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:35.891840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:35.891840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:35.891840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:35.891840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 387/785] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:35.901866 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698b54c-bd3c-4ccc-a932-73d91ac7b52a None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:35.936443 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:36.018750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd5e19-e9c5-4123-9963-47533b97eb6c None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:36.019153 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd5e19-e9c5-4123-9963-47533b97eb6c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:36.019332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd5e19-e9c5-4123-9963-47533b97eb6c None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:36.021016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 388/786] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:31:36.027754 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdfb38-e0ad-43ef-bced-70aab1f3015a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:36.028110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdfb38-e0ad-43ef-bced-70aab1f3015a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:36.028336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdfb38-e0ad-43ef-bced-70aab1f3015a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:36.049826 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-30cdfb38-e0ad-43ef-bced-70aab1f3015a None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:36.051046 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-30cdfb38-e0ad-43ef-bced-70aab1f3015a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:36.083284 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698b54c-bd3c-4ccc-a932-73d91ac7b52a None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:36.161663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 389/787] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:31:36.170042 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd7ce0-f607-46d1-b6c3-4142a70e0e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:36.170813 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd7ce0-f607-46d1-b6c3-4142a70e0e2d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:36.171097 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd7ce0-f607-46d1-b6c3-4142a70e0e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:36.171351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd7ce0-f607-46d1-b6c3-4142a70e0e2d None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:36.172048 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd7ce0-f607-46d1-b6c3-4142a70e0e2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:36.172351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd7ce0-f607-46d1-b6c3-4142a70e0e2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:36.173814 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd7ce0-f607-46d1-b6c3-4142a70e0e2d None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:36.202704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 399/788] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:31:36.211703 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdfb38-e0ad-43ef-bced-70aab1f3015a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:36.368474 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:36.453664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdfb38-e0ad-43ef-bced-70aab1f3015a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:36.491399 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808899e2-e8ff-4980-827d-2768000c09b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:36.492075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808899e2-e8ff-4980-827d-2768000c09b6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:36.492351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808899e2-e8ff-4980-827d-2768000c09b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:36.492601 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808899e2-e8ff-4980-827d-2768000c09b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:36.500918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808899e2-e8ff-4980-827d-2768000c09b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:36.502682 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808899e2-e8ff-4980-827d-2768000c09b6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:36.523028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-808899e2-e8ff-4980-827d-2768000c09b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:31:36.523028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:36.523028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:36.523028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:36.523028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:36.523028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:36.523028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:36.523028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:36.523028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:36.523028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:36.523028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:36.523028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:36.523028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:36.523028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:36.523028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:36.523028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:36.524108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:36.525093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:36.525093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:36.525093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:36.525093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:36.525093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:36.525093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:36.525093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:36.525093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:36.525093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:31:36.525093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:36.525647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 390/789] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Aug 30 14:31:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:36.532238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd7ce0-f607-46d1-b6c3-4142a70e0e2d None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:36.595581 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3d503-44a9-4716-b3ac-3385643fbe5a None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:36.595922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3d503-44a9-4716-b3ac-3385643fbe5a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:36.596168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3d503-44a9-4716-b3ac-3385643fbe5a None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:36.597696 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 391/790] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:31:36.613195 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca3c44-3ea0-461f-97a1-a23d68c88158 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:36.615718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca3c44-3ea0-461f-97a1-a23d68c88158 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:36.616024 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca3c44-3ea0-461f-97a1-a23d68c88158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:36.643287 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-67ca3c44-3ea0-461f-97a1-a23d68c88158 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:36.644958 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-67ca3c44-3ea0-461f-97a1-a23d68c88158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:36.691586 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:36.726929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd7ce0-f607-46d1-b6c3-4142a70e0e2d None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:36.779126 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 392/791] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:31:36.781717 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca3c44-3ea0-461f-97a1-a23d68c88158 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:36.800307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3895e0-3450-429b-b313-958d2e34a89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:36.801862 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3895e0-3450-429b-b313-958d2e34a89b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:36.802667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3895e0-3450-429b-b313-958d2e34a89b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:36.803428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3895e0-3450-429b-b313-958d2e34a89b None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:36.805059 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3895e0-3450-429b-b313-958d2e34a89b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:36.806848 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3895e0-3450-429b-b313-958d2e34a89b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:36.812384 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3895e0-3450-429b-b313-958d2e34a89b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:36.868413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 400/792] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Aug 30 14:31:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:36.874954 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3895e0-3450-429b-b313-958d2e34a89b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:36.895577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:36.963032 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca3c44-3ea0-461f-97a1-a23d68c88158 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:37.010361 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb05d8-a302-41ce-ba5d-38db56ddc2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:37.011515 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb05d8-a302-41ce-ba5d-38db56ddc2ee None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:37.012113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb05d8-a302-41ce-ba5d-38db56ddc2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:37.012555 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb05d8-a302-41ce-ba5d-38db56ddc2ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:37.025683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3895e0-3450-429b-b313-958d2e34a89b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:37.032254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb05d8-a302-41ce-ba5d-38db56ddc2ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:37.038808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-8feb05d8-a302-41ce-ba5d-38db56ddc2ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:37.038808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:37.038808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:37.038808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:37.038808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:37.038808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:37.038808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:37.038808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:37.038808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:37.038808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:37.038808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:37.038808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:37.038808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:37.038808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:37.038808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:37.038808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:37.040058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:37.040058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:37.040058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:37.040058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:37.040058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:37.040058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:37.040058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:37.040058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:37.040058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:37.040058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:37.040058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:37.040058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:37.040058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:37.040058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:37.040058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:37.040058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:37.041125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:37.041125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:37.041125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:37.041125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:37.041125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:37.041125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:37.041125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:37.041125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:37.041125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:37.041125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:37.041125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:37.041125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:37.044726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 393/793] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:37.046757 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe92f50-e70f-49af-ba1b-4bda93c3f9fd None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:37.047249 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe92f50-e70f-49af-ba1b-4bda93c3f9fd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:37.048242 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe92f50-e70f-49af-ba1b-4bda93c3f9fd None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:37.051264 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 394/794] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:37.053038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe92f50-e70f-49af-ba1b-4bda93c3f9fd None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:37.078539 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276057f-4e15-4255-ad34-127861cefaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:37.079354 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276057f-4e15-4255-ad34-127861cefaaf None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:37.079741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276057f-4e15-4255-ad34-127861cefaaf None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:37.080264 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276057f-4e15-4255-ad34-127861cefaaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:37.112780 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:37.190398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276057f-4e15-4255-ad34-127861cefaaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:37.192693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276057f-4e15-4255-ad34-127861cefaaf None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:37.201221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe92f50-e70f-49af-ba1b-4bda93c3f9fd None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:37.203187 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:37.247647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8a3839-2b85-4f50-9751-20c471e65b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:37.249287 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a3839-2b85-4f50-9751-20c471e65b2a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:37.249760 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a3839-2b85-4f50-9751-20c471e65b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:37.250085 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a3839-2b85-4f50-9751-20c471e65b2a None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:37.250847 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8a3839-2b85-4f50-9751-20c471e65b2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:37.251214 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8a3839-2b85-4f50-9751-20c471e65b2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:37.252724 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8a3839-2b85-4f50-9751-20c471e65b2a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:37.268163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 395/795] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:31:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:37.270692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ae81f-88ec-4886-a797-eba41223ec45 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:37.271531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ae81f-88ec-4886-a797-eba41223ec45 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:37.272338 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ae81f-88ec-4886-a797-eba41223ec45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:37.304197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-767ae81f-88ec-4886-a797-eba41223ec45 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:37.306283 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-767ae81f-88ec-4886-a797-eba41223ec45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:37.360440 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 401/796] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:31:37.361734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276057f-4e15-4255-ad34-127861cefaaf None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:37.413591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 396/797] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:37.414875 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ae81f-88ec-4886-a797-eba41223ec45 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:37.441049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:37.536347 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276057f-4e15-4255-ad34-127861cefaaf None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:37.581345 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:37.590269 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ae81f-88ec-4886-a797-eba41223ec45 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:37.595723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befef69-0e25-486f-9052-8391ba626ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:37.596702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befef69-0e25-486f-9052-8391ba626ccd None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:37.597193 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befef69-0e25-486f-9052-8391ba626ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:37.597664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befef69-0e25-486f-9052-8391ba626ccd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:37.612956 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7befef69-0e25-486f-9052-8391ba626ccd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:37.615164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7befef69-0e25-486f-9052-8391ba626ccd None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:37.632923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad8ee4-e33d-4a84-b5eb-091a856e3587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:37.633823 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad8ee4-e33d-4a84-b5eb-091a856e3587 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:37.634097 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad8ee4-e33d-4a84-b5eb-091a856e3587 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:37.634363 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad8ee4-e33d-4a84-b5eb-091a856e3587 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:37.635013 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad8ee4-e33d-4a84-b5eb-091a856e3587 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:37.636445 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad8ee4-e33d-4a84-b5eb-091a856e3587 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:37.637103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-7befef69-0e25-486f-9052-8391ba626ccd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:37.637103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:37.637103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:37.637103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:37.637103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:37.637103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:37.637103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:37.637103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:37.637103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:37.637103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:37.637103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:37.637103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:37.637103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:37.637103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:37.637103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:37.637103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:37.638056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:37.638056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:37.638056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:37.638056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:37.638056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:37.638056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:37.638056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:37.638056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:37.638056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:37.638056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:37.638056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:37.638056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:37.638056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:37.638056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:37.638056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:37.638056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:37.638056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:37.639534 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:37.639534 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:37.639534 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:37.639713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 402/798] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:37.641384 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befef69-0e25-486f-9052-8391ba626ccd None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:37.652899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-a1ad8ee4-e33d-4a84-b5eb-091a856e3587 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:31:37.652899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:37.652899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:31:37.652899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:31:37.652899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:31:37.652899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:31:37.652899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:31:37.652899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:37.652899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:31:37.652899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:37.652899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:37.652899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:37.652899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:37.652899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:37.652899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:37.652899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:37.652899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:37.654992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:37.654992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:37.654992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:37.654992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:37.654992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:37.654992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:37.654992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:31:37.654992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:31:37.654992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:31:37.654992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:31:37.654992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:37.654992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:37.654992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:37.654992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:37.654992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:37.654992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:37.656830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:37.656830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:37.656830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:37.656830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:37.656830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:37.656830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:37.656830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:37.656830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:37.656830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:37.656830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:37.656830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:31:37.656830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:37.656830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:31:37.656830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:37.668059 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 397/799] 10.222.0.25 () {56 vars in 1129 bytes} [Tue Aug 30 14:31:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:37.668059 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad8ee4-e33d-4a84-b5eb-091a856e3587 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:37.787406 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad8ee4-e33d-4a84-b5eb-091a856e3587 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:37.812266 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:37.830858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befef69-0e25-486f-9052-8391ba626ccd None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:37.832739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de632e4-1b65-4a6c-8813-211621555628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:37.833521 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de632e4-1b65-4a6c-8813-211621555628 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:37.834177 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de632e4-1b65-4a6c-8813-211621555628 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:37.834609 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de632e4-1b65-4a6c-8813-211621555628 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:37.835555 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de632e4-1b65-4a6c-8813-211621555628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:37.836055 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de632e4-1b65-4a6c-8813-211621555628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:37.837898 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de632e4-1b65-4a6c-8813-211621555628 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:37.860055 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 398/800] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:37.860055 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de632e4-1b65-4a6c-8813-211621555628 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:37.886989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:37.892775 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62108bc8-b111-45bc-8c6e-67bbe542e369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:37.893427 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62108bc8-b111-45bc-8c6e-67bbe542e369 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:37.894055 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62108bc8-b111-45bc-8c6e-67bbe542e369 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:37.894055 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62108bc8-b111-45bc-8c6e-67bbe542e369 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:37.909102 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62108bc8-b111-45bc-8c6e-67bbe542e369 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:37.910665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62108bc8-b111-45bc-8c6e-67bbe542e369 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:37.930966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-62108bc8-b111-45bc-8c6e-67bbe542e369 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:37.930966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:37.930966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:37.930966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:37.930966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:37.930966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:37.930966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:37.930966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:37.930966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:37.930966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:37.930966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:37.930966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:37.930966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:37.930966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:37.930966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:37.930966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:37.932096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:37.932096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:37.932096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:37.932096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:37.932096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:37.932096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:37.932096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:37.932096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:37.932096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:37.932096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:37.932096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:37.932096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:37.932096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:37.932096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:37.932096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:37.932096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:37.932096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:37.933250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:37.933250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:37.933250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:37.933250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 403/801] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:37.933250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62108bc8-b111-45bc-8c6e-67bbe542e369 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:38.028778 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de632e4-1b65-4a6c-8813-211621555628 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:38.047917 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62108bc8-b111-45bc-8c6e-67bbe542e369 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:38.069857 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c1201-a1b2-42bf-870d-1469438a3838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:38.070487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c1201-a1b2-42bf-870d-1469438a3838 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:38.071257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c1201-a1b2-42bf-870d-1469438a3838 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:38.071531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c1201-a1b2-42bf-870d-1469438a3838 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:38.072238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c1201-a1b2-42bf-870d-1469438a3838 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:38.072534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c1201-a1b2-42bf-870d-1469438a3838 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:38.073952 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c1201-a1b2-42bf-870d-1469438a3838 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:38.079375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 399/802] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Aug 30 14:31:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:38.080282 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c1201-a1b2-42bf-870d-1469438a3838 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:38.086293 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46aa4bf-54b4-47fe-b425-fc4cbf2b9ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:38.087023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46aa4bf-54b4-47fe-b425-fc4cbf2b9ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:38.087275 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46aa4bf-54b4-47fe-b425-fc4cbf2b9ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:38.087544 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46aa4bf-54b4-47fe-b425-fc4cbf2b9ba9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:38.099467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46aa4bf-54b4-47fe-b425-fc4cbf2b9ba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:38.104810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-f46aa4bf-54b4-47fe-b425-fc4cbf2b9ba9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:38.104810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:38.104810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:38.104810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:38.104810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:38.104810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:38.104810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:38.104810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:38.104810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:38.104810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:38.104810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:38.104810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:38.104810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:38.104810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:38.104810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:38.104810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:38.106553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:38.106553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:38.106553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:38.106553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:38.106553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:38.106553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:38.106553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:38.106553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:38.106553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:38.106553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:38.106553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:38.106553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:38.106553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:38.106553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:38.106553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:38.106553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:38.107960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:38.107960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:38.107960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:38.107960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:38.107960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:38.107960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:38.107960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:38.107960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:38.107960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:38.107960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:38.107960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:38.107960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:38.107960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 404/803] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:31:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:38.107960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46aa4bf-54b4-47fe-b425-fc4cbf2b9ba9 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:38.110548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:38.189690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:38.219537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c1201-a1b2-42bf-870d-1469438a3838 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:38.222625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46aa4bf-54b4-47fe-b425-fc4cbf2b9ba9 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:38.261062 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c11c91-3d26-4295-b79e-0521c622c6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:38.261709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c11c91-3d26-4295-b79e-0521c622c6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:38.261968 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c11c91-3d26-4295-b79e-0521c622c6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:38.262258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c11c91-3d26-4295-b79e-0521c622c6a0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:38.269821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3172f-d820-4d86-a116-801ef0bffcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:38.270021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c11c91-3d26-4295-b79e-0521c622c6a0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:38.270887 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3172f-d820-4d86-a116-801ef0bffcc2 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:38.271406 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3172f-d820-4d86-a116-801ef0bffcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:38.271526 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c11c91-3d26-4295-b79e-0521c622c6a0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:38.271882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3172f-d820-4d86-a116-801ef0bffcc2 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:38.273092 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3172f-d820-4d86-a116-801ef0bffcc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:38.274098 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3172f-d820-4d86-a116-801ef0bffcc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:38.275747 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3172f-d820-4d86-a116-801ef0bffcc2 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:38.284022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-40c11c91-3d26-4295-b79e-0521c622c6a0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:31:38.284022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:38.284022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:38.284022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:38.284022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:38.284022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:38.284022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:38.284022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:38.284022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:38.284022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:38.284022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:38.284022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:38.284022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:38.284022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:38.284022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:38.284022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:38.285073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:38.286029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:38.286029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:38.286029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:38.286029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:38.286029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:38.286029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:38.286029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:38.286029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:38.286029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:31:38.286029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:38.286029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 400/804] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:38.286633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8ab31-4054-4ee3-ace3-60ec372a86e5 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:38.286633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8ab31-4054-4ee3-ace3-60ec372a86e5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:38.287224 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8ab31-4054-4ee3-ace3-60ec372a86e5 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:38.291298 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 401/805] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:38.291298 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8ab31-4054-4ee3-ace3-60ec372a86e5 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:38.302375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 405/806] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:38.302375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3172f-d820-4d86-a116-801ef0bffcc2 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:38.376878 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:38.416202 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8ab31-4054-4ee3-ace3-60ec372a86e5 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:38.422350 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3172f-d820-4d86-a116-801ef0bffcc2 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:38.455593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c477b8e-303b-4d68-92e0-bbfbba1f7c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:38.456266 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c477b8e-303b-4d68-92e0-bbfbba1f7c3e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:38.456554 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c477b8e-303b-4d68-92e0-bbfbba1f7c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:38.457175 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c477b8e-303b-4d68-92e0-bbfbba1f7c3e None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:38.457815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c477b8e-303b-4d68-92e0-bbfbba1f7c3e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:38.458113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c477b8e-303b-4d68-92e0-bbfbba1f7c3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:38.459552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c477b8e-303b-4d68-92e0-bbfbba1f7c3e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:38.464647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34d441-08e7-451f-9fa2-88e779d09706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:38.465237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 402/807] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Aug 30 14:31:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:38.465347 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34d441-08e7-451f-9fa2-88e779d09706 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:38.465544 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34d441-08e7-451f-9fa2-88e779d09706 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:38.465796 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34d441-08e7-451f-9fa2-88e779d09706 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:38.466054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c477b8e-303b-4d68-92e0-bbfbba1f7c3e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:38.466473 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34d441-08e7-451f-9fa2-88e779d09706 None admin] RBAC: Adding query filter params (name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:38.466769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34d441-08e7-451f-9fa2-88e779d09706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:38.468214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34d441-08e7-451f-9fa2-88e779d09706 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:38.477135 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 406/808] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:31:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:38.478399 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd9bd0-034a-4a5f-a7b1-0eac6898efc6 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:38.478678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd9bd0-034a-4a5f-a7b1-0eac6898efc6 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:38.478949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd9bd0-034a-4a5f-a7b1-0eac6898efc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:38.499339 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-eddd9bd0-034a-4a5f-a7b1-0eac6898efc6 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:38.500637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-eddd9bd0-034a-4a5f-a7b1-0eac6898efc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:38.608324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:38.609586 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 407/809] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:38.611075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd9bd0-034a-4a5f-a7b1-0eac6898efc6 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:38.699917 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:38.723951 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c477b8e-303b-4d68-92e0-bbfbba1f7c3e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:38.780898 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd9bd0-034a-4a5f-a7b1-0eac6898efc6 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:38.783856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6790c-640d-4dd2-b37c-d1bfeb86eac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:38.784857 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6790c-640d-4dd2-b37c-d1bfeb86eac4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:38.785312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6790c-640d-4dd2-b37c-d1bfeb86eac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:38.785761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6790c-640d-4dd2-b37c-d1bfeb86eac4 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:38.786724 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6790c-640d-4dd2-b37c-d1bfeb86eac4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:38.787145 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6790c-640d-4dd2-b37c-d1bfeb86eac4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:38.789695 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6790c-640d-4dd2-b37c-d1bfeb86eac4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:38.818688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 403/810] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Aug 30 14:31:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:38.820061 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6790c-640d-4dd2-b37c-d1bfeb86eac4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:38.849385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93cc8e-4db4-4d8d-a9c6-fc75d4674fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:38.850457 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93cc8e-4db4-4d8d-a9c6-fc75d4674fcc None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:38.850950 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93cc8e-4db4-4d8d-a9c6-fc75d4674fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:38.851407 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93cc8e-4db4-4d8d-a9c6-fc75d4674fcc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:38.862357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93cc8e-4db4-4d8d-a9c6-fc75d4674fcc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:38.864037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93cc8e-4db4-4d8d-a9c6-fc75d4674fcc None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:38.887649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-de93cc8e-4db4-4d8d-a9c6-fc75d4674fcc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:38.887649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:38.887649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:38.887649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:38.887649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:38.887649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:38.887649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:38.887649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:38.887649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:38.887649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:38.887649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:38.887649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:38.887649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:38.887649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:38.887649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:38.889047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:38.889047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:38.889047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:38.889047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:38.889047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:38.889047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:38.889047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:38.889047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:38.889047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:38.889047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:38.889047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:38.889047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:38.889047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:38.889047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:38.889047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:38.889047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:38.889047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:38.890252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:38.890252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:38.890252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:38.890252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:38.890252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:38.890252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:38.890252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:38.890252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:38.890842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 408/811] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:38.892228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93cc8e-4db4-4d8d-a9c6-fc75d4674fcc None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:38.908037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:38.949563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6790c-640d-4dd2-b37c-d1bfeb86eac4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:38.989703 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb806dd-f2bf-4718-a649-d19a6a83702c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:38.990661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb806dd-f2bf-4718-a649-d19a6a83702c None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:38.990929 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb806dd-f2bf-4718-a649-d19a6a83702c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:38.991251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb806dd-f2bf-4718-a649-d19a6a83702c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:38.991875 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb806dd-f2bf-4718-a649-d19a6a83702c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:38.993298 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb806dd-f2bf-4718-a649-d19a6a83702c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:39.018783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 404/812] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:31:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:31:39.019646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb806dd-f2bf-4718-a649-d19a6a83702c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:39.055027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93cc8e-4db4-4d8d-a9c6-fc75d4674fcc None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:39.081257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:39.131450 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310f7f2-e503-4b1d-af6a-24cb17f6b52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:39.132179 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310f7f2-e503-4b1d-af6a-24cb17f6b52f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:39.132653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310f7f2-e503-4b1d-af6a-24cb17f6b52f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:39.133123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310f7f2-e503-4b1d-af6a-24cb17f6b52f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:39.151922 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310f7f2-e503-4b1d-af6a-24cb17f6b52f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:39.154760 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310f7f2-e503-4b1d-af6a-24cb17f6b52f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:39.172586 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb806dd-f2bf-4718-a649-d19a6a83702c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:39.182987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-6310f7f2-e503-4b1d-af6a-24cb17f6b52f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:39.182987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:39.182987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:39.182987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:39.182987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:39.182987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:39.182987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:39.182987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:39.182987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:39.182987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:39.182987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:39.182987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:39.182987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:39.182987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:39.182987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:39.182987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:39.184821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:39.184821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:39.184821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:39.184821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:39.184821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:39.184821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:39.184821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:39.184821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:39.184821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:39.184821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:39.184821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:39.184821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:39.184821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:39.184821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:39.184821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:39.184821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:39.184821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:39.186303 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:39.186303 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:39.186303 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:39.186303 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 409/813] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:39.186303 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310f7f2-e503-4b1d-af6a-24cb17f6b52f None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:39.210671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:39.237390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30a877-76dd-4394-a2bf-440b706c1d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:39.238362 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30a877-76dd-4394-a2bf-440b706c1d8d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:39.238836 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30a877-76dd-4394-a2bf-440b706c1d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:39.239263 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30a877-76dd-4394-a2bf-440b706c1d8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:39.257212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30a877-76dd-4394-a2bf-440b706c1d8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:39.259685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30a877-76dd-4394-a2bf-440b706c1d8d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:39.275918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-8a30a877-76dd-4394-a2bf-440b706c1d8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:39.275918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:39.275918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:39.275918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:39.275918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:39.275918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:39.275918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:39.275918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:39.275918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:39.275918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:39.275918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:39.275918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:39.275918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:39.275918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:39.275918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:39.275918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:39.276976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:39.276976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:39.276976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:39.276976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:39.276976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:39.276976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:39.276976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:39.276976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:39.276976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:39.276976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:39.276976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:39.276976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:39.276976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:39.276976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:39.276976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:39.276976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:39.276976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:39.277855 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:39.277855 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:39.277855 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:39.277855 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 405/814] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:39.278084 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30a877-76dd-4394-a2bf-440b706c1d8d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:39.335456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310f7f2-e503-4b1d-af6a-24cb17f6b52f None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:39.376484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80dbc3-3813-4d69-9fc2-dfb8d8f23d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:39.377573 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80dbc3-3813-4d69-9fc2-dfb8d8f23d3b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:39.378073 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80dbc3-3813-4d69-9fc2-dfb8d8f23d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:39.378556 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80dbc3-3813-4d69-9fc2-dfb8d8f23d3b None admin] PATH_INFO: `/v3/services/network` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:39.379512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa80dbc3-3813-4d69-9fc2-dfb8d8f23d3b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:39.381287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa80dbc3-3813-4d69-9fc2-dfb8d8f23d3b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:39.392538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-aa80dbc3-3813-4d69-9fc2-dfb8d8f23d3b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:31:39.392538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:39.392538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:31:39.392538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:31:39.392538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:31:39.392538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:31:39.392538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:31:39.392538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:39.392538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:31:39.392538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:39.392538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:39.392538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:39.392538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:39.392538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:39.392538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:39.392538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:39.392538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:39.393806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:39.393806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:39.393806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:39.393806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:39.393806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:39.393806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:39.393806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:31:39.393806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:31:39.393806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:31:39.393806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:31:39.393806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:39.393806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:39.393806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:39.393806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:39.393806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:39.393806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:39.394843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:39.394843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:39.394843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:39.394843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:39.394843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:39.394843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:39.394843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:39.394843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:39.394843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:39.394843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:39.394843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:31:39.394843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:39.394843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:31:39.394843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:39.397255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 410/815] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:39 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:39.398392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80dbc3-3813-4d69-9fc2-dfb8d8f23d3b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:39.403969 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30a877-76dd-4394-a2bf-440b706c1d8d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:39.436477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:39.449111 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930fb83-71d6-4bc6-a941-15a32e280820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:39.449754 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930fb83-71d6-4bc6-a941-15a32e280820 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:39.450078 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930fb83-71d6-4bc6-a941-15a32e280820 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:39.450292 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930fb83-71d6-4bc6-a941-15a32e280820 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:39.451026 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930fb83-71d6-4bc6-a941-15a32e280820 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:39.451415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930fb83-71d6-4bc6-a941-15a32e280820 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:39.453254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930fb83-71d6-4bc6-a941-15a32e280820 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:39.473963 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 406/816] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:39.473963 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930fb83-71d6-4bc6-a941-15a32e280820 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:39.518175 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:39.541748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80dbc3-3813-4d69-9fc2-dfb8d8f23d3b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:39.602812 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83d1aa-02de-4754-90a1-8dbe7592e20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:39.603717 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83d1aa-02de-4754-90a1-8dbe7592e20c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:39.604155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83d1aa-02de-4754-90a1-8dbe7592e20c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:39.604565 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83d1aa-02de-4754-90a1-8dbe7592e20c None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:39.605552 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83d1aa-02de-4754-90a1-8dbe7592e20c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:39.606008 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83d1aa-02de-4754-90a1-8dbe7592e20c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:39.610147 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83d1aa-02de-4754-90a1-8dbe7592e20c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:39.633146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 411/817] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:39.633146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83d1aa-02de-4754-90a1-8dbe7592e20c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:39.641458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930fb83-71d6-4bc6-a941-15a32e280820 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:39.689061 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57bbb01-ca86-4b94-a0a3-f56f8846c4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:39.690038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bbb01-ca86-4b94-a0a3-f56f8846c4dc None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:39.690490 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bbb01-ca86-4b94-a0a3-f56f8846c4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:39.691013 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bbb01-ca86-4b94-a0a3-f56f8846c4dc None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:39.691834 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57bbb01-ca86-4b94-a0a3-f56f8846c4dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:39.692287 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57bbb01-ca86-4b94-a0a3-f56f8846c4dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:39.694190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57bbb01-ca86-4b94-a0a3-f56f8846c4dc None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:39.707830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 407/818] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:39.712874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57bbb01-ca86-4b94-a0a3-f56f8846c4dc None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:39.756554 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:39.780329 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83d1aa-02de-4754-90a1-8dbe7592e20c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:39.833438 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07c8c0-7b9b-4c6f-85fb-2f1cbad0727e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:39.834071 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07c8c0-7b9b-4c6f-85fb-2f1cbad0727e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:39.834893 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07c8c0-7b9b-4c6f-85fb-2f1cbad0727e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:39.835168 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07c8c0-7b9b-4c6f-85fb-2f1cbad0727e None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:39.835823 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07c8c0-7b9b-4c6f-85fb-2f1cbad0727e None admin] RBAC: Adding query filter params (name=network) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:39.836167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07c8c0-7b9b-4c6f-85fb-2f1cbad0727e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:39.838011 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07c8c0-7b9b-4c6f-85fb-2f1cbad0727e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:39.852077 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 412/819] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Aug 30 14:31:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:39.852077 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07c8c0-7b9b-4c6f-85fb-2f1cbad0727e None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:39.862746 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:39.879726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57bbb01-ca86-4b94-a0a3-f56f8846c4dc None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:39.936204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54237bfa-a432-4483-85a9-468f1926a1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:39.937053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54237bfa-a432-4483-85a9-468f1926a1da None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:39.937446 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54237bfa-a432-4483-85a9-468f1926a1da None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:39.937833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54237bfa-a432-4483-85a9-468f1926a1da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:39.938877 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54237bfa-a432-4483-85a9-468f1926a1da None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=cc95bbdafb0c4f34ad0662bc2e3ba4ec) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:39.939376 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54237bfa-a432-4483-85a9-468f1926a1da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:39.941631 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54237bfa-a432-4483-85a9-468f1926a1da None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:39.959334 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 408/820] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:39 2022] GET /identity/v3/role_assignments?user.id=cc95bbdafb0c4f34ad0662bc2e3ba4ec&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:39.959334 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54237bfa-a432-4483-85a9-468f1926a1da None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:40.000034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07c8c0-7b9b-4c6f-85fb-2f1cbad0727e None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:40.048379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2766297-bf02-4c1e-9493-ab7ac3ffd30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:40.049391 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2766297-bf02-4c1e-9493-ab7ac3ffd30f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:40.049791 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2766297-bf02-4c1e-9493-ab7ac3ffd30f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:40.050131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2766297-bf02-4c1e-9493-ab7ac3ffd30f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:40.060973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2766297-bf02-4c1e-9493-ab7ac3ffd30f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:40.062892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2766297-bf02-4c1e-9493-ab7ac3ffd30f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:40.159468 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54237bfa-a432-4483-85a9-468f1926a1da None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:40.213713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbaf61b-4ae4-409d-ace4-124cb6ece358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:40.215227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbaf61b-4ae4-409d-ace4-124cb6ece358 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:40.215763 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbaf61b-4ae4-409d-ace4-124cb6ece358 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:40.216299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbaf61b-4ae4-409d-ace4-124cb6ece358 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:40.225308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-e2766297-bf02-4c1e-9493-ab7ac3ffd30f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:31:40.225308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:40.225308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:40.225308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:40.225308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:40.225308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:40.225308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:40.225308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:40.225308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:40.225308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:40.225308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:40.225308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:40.225308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:40.225308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:40.225308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:40.225308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:40.227199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:40.228368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:40.228368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:40.228368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:40.228368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:40.228368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:40.228368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:40.228368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:40.228368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:40.228368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:31:40.228368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:40.228368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 413/821] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Aug 30 14:31:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:40.228368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2766297-bf02-4c1e-9493-ab7ac3ffd30f None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:40.238483 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbaf61b-4ae4-409d-ace4-124cb6ece358 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:40.240011 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbaf61b-4ae4-409d-ace4-124cb6ece358 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:40.247476 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:40.255828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-7fbaf61b-4ae4-409d-ace4-124cb6ece358 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:40.255828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:40.255828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:40.255828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:40.255828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:40.255828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:40.255828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:40.255828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:40.255828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:40.255828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:40.255828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:40.255828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:40.255828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:40.255828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:40.255828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:40.257227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:40.257227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:40.257227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:40.257227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:40.257227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:40.257227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:40.257227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:40.257227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:40.257227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:40.257227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:40.257227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:40.257227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:40.257227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:40.257227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:40.257227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:40.257227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:40.257227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:40.258487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:40.258487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:40.258487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:40.258487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:40.258487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:40.258487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:40.258487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:40.258487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:40.258487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 409/822] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:40.258487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbaf61b-4ae4-409d-ace4-124cb6ece358 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:40.298142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:40.430615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2766297-bf02-4c1e-9493-ab7ac3ffd30f None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:40.450495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbaf61b-4ae4-409d-ace4-124cb6ece358 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:40.480429 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef742c3f-0c0b-4af2-9365-b56cab9ce575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:40.481081 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef742c3f-0c0b-4af2-9365-b56cab9ce575 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:40.481358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef742c3f-0c0b-4af2-9365-b56cab9ce575 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:40.481629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef742c3f-0c0b-4af2-9365-b56cab9ce575 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:40.482336 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef742c3f-0c0b-4af2-9365-b56cab9ce575 None admin] RBAC: Adding query filter params (type=network) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:40.482696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef742c3f-0c0b-4af2-9365-b56cab9ce575 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:40.484409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef742c3f-0c0b-4af2-9365-b56cab9ce575 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:40.497461 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 414/823] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Aug 30 14:31:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:40.497461 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef742c3f-0c0b-4af2-9365-b56cab9ce575 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:40.540334 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd85b30-a01a-4276-bf4d-a6a103f0bc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:40.542642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd85b30-a01a-4276-bf4d-a6a103f0bc4e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:40.543357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd85b30-a01a-4276-bf4d-a6a103f0bc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:40.544339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd85b30-a01a-4276-bf4d-a6a103f0bc4e None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:40.545887 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd85b30-a01a-4276-bf4d-a6a103f0bc4e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:40.546588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd85b30-a01a-4276-bf4d-a6a103f0bc4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:40.551373 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd85b30-a01a-4276-bf4d-a6a103f0bc4e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:40.594801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 410/824] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Aug 30 14:31:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:40.596586 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd85b30-a01a-4276-bf4d-a6a103f0bc4e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:40.616805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:40.661326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef742c3f-0c0b-4af2-9365-b56cab9ce575 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:40.718997 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4721294-1a69-460d-bb07-b67c19b42f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:40.719774 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4721294-1a69-460d-bb07-b67c19b42f5d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:40.720173 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4721294-1a69-460d-bb07-b67c19b42f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:40.720530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4721294-1a69-460d-bb07-b67c19b42f5d None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:40.721093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:40.721494 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4721294-1a69-460d-bb07-b67c19b42f5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:40.722338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4721294-1a69-460d-bb07-b67c19b42f5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:40.724190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4721294-1a69-460d-bb07-b67c19b42f5d None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:40.739861 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 415/825] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:40.740826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4721294-1a69-460d-bb07-b67c19b42f5d None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:40.781569 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd85b30-a01a-4276-bf4d-a6a103f0bc4e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:40.823090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a991b7d-07be-4dbb-8245-fee9cdb94486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:40.823715 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a991b7d-07be-4dbb-8245-fee9cdb94486 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:40.823986 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a991b7d-07be-4dbb-8245-fee9cdb94486 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:40.824279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a991b7d-07be-4dbb-8245-fee9cdb94486 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:40.825000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a991b7d-07be-4dbb-8245-fee9cdb94486 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:40.826534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a991b7d-07be-4dbb-8245-fee9cdb94486 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:40.856603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 411/826] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:31:40 2022] POST /identity/v3/endpoints => generated 337 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:31:40.857841 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a991b7d-07be-4dbb-8245-fee9cdb94486 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:40.943544 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4721294-1a69-460d-bb07-b67c19b42f5d None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:41.014513 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:41.027047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412ab94-4f97-46cf-a32e-26eb2e574fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:41.028139 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412ab94-4f97-46cf-a32e-26eb2e574fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:41.028600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412ab94-4f97-46cf-a32e-26eb2e574fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:41.029065 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412ab94-4f97-46cf-a32e-26eb2e574fd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:41.053210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412ab94-4f97-46cf-a32e-26eb2e574fd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:41.055981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412ab94-4f97-46cf-a32e-26eb2e574fd0 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:41.076123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-6412ab94-4f97-46cf-a32e-26eb2e574fd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:41.076123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:41.076123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:41.076123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:41.076123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:41.076123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:41.076123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:41.076123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:41.076123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:41.076123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:41.076123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:41.076123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:41.076123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:41.076123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:41.076123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:41.076123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:41.078935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:41.078935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:41.078935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:41.078935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:41.078935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:41.078935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:41.078935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:41.078935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:41.078935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:41.078935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:41.078935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:41.078935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:41.078935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:41.078935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:41.078935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:41.078935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:41.078935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:41.081097 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:41.081097 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:41.081097 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:41.081476 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a991b7d-07be-4dbb-8245-fee9cdb94486 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:41.088027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 416/827] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:41.088027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412ab94-4f97-46cf-a32e-26eb2e574fd0 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:41.148861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:41.158158 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8765559-a001-4143-9670-992046bff0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:41.159038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8765559-a001-4143-9670-992046bff0e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:41.159537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8765559-a001-4143-9670-992046bff0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:41.159859 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8765559-a001-4143-9670-992046bff0e4 None admin] PATH_INFO: `/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/d5d98f5adfaf410580481417e7ba1638/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:41.164037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8765559-a001-4143-9670-992046bff0e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9c57a581324217ad2e64a0c9c7fdee, user_id=d5d98f5adfaf410580481417e7ba1638, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:41.165514 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8765559-a001-4143-9670-992046bff0e4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:41.181339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 412/828] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:31:40 2022] PUT /identity/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/d5d98f5adfaf410580481417e7ba1638/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:31:41.201698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412ab94-4f97-46cf-a32e-26eb2e574fd0 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:41.243171 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62ee5f-bb7f-4449-ae05-cc7aaa9cf1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:41.243820 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62ee5f-bb7f-4449-ae05-cc7aaa9cf1fd None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:41.244119 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62ee5f-bb7f-4449-ae05-cc7aaa9cf1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:41.244386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62ee5f-bb7f-4449-ae05-cc7aaa9cf1fd None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:41.245078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62ee5f-bb7f-4449-ae05-cc7aaa9cf1fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:41.245378 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62ee5f-bb7f-4449-ae05-cc7aaa9cf1fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:41.246818 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62ee5f-bb7f-4449-ae05-cc7aaa9cf1fd None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:41.266790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 417/829] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:41.269874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8765559-a001-4143-9670-992046bff0e4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:41.329988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82592dff-1173-4d81-af77-33c2e902f7e6 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:41.330314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82592dff-1173-4d81-af77-33c2e902f7e6 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:41.330515 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82592dff-1173-4d81-af77-33c2e902f7e6 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:41.332682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 418/830] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:31:41.341348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5a917-2667-412b-816e-5f7e4071732d None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:41.341664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5a917-2667-412b-816e-5f7e4071732d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:41.342135 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5a917-2667-412b-816e-5f7e4071732d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:41.368947 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a6c5a917-2667-412b-816e-5f7e4071732d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:41.370136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a6c5a917-2667-412b-816e-5f7e4071732d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:41.405071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8765559-a001-4143-9670-992046bff0e4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:41.428393 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:41.465760 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc885ca-7ee2-4fae-a705-16b5c9520557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:41.466930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc885ca-7ee2-4fae-a705-16b5c9520557 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:41.467388 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc885ca-7ee2-4fae-a705-16b5c9520557 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:41.467608 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 419/831] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:31:41.467875 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc885ca-7ee2-4fae-a705-16b5c9520557 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:41.484107 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc885ca-7ee2-4fae-a705-16b5c9520557 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:41.484107 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc885ca-7ee2-4fae-a705-16b5c9520557 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:41.507064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-1fc885ca-7ee2-4fae-a705-16b5c9520557 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:41.507064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:41.507064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:41.507064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:41.507064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:41.507064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:41.507064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:41.507064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:41.507064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:41.507064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:41.507064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:41.507064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:41.507064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:41.507064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:41.507064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:41.508879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:41.508879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:41.508879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:41.508879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:41.508879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:41.508879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:41.508879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:41.508879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:41.508879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:41.508879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:41.508879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:41.508879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:41.508879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:41.508879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:41.508879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:41.508879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:41.508879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:41.510681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:41.510681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:41.510681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:41.510681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:41.510681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:41.510681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:41.510681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:41.510681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:41.510681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 413/832] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:41.516707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5a917-2667-412b-816e-5f7e4071732d None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:41.646540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5a917-2667-412b-816e-5f7e4071732d None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:41.685370 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74b74c-aa79-4013-af58-17648a6060a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:41.686338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74b74c-aa79-4013-af58-17648a6060a1 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:41.686647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74b74c-aa79-4013-af58-17648a6060a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:41.686918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74b74c-aa79-4013-af58-17648a6060a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:41.687662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74b74c-aa79-4013-af58-17648a6060a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:41.687970 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74b74c-aa79-4013-af58-17648a6060a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:41.689463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74b74c-aa79-4013-af58-17648a6060a1 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:41.704846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 420/833] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:41.711106 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc885ca-7ee2-4fae-a705-16b5c9520557 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:41.753465 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74b74c-aa79-4013-af58-17648a6060a1 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:41.770177 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:41.795589 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:41.837196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc885ca-7ee2-4fae-a705-16b5c9520557 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:41.880333 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bf54b-efdc-4060-a5b0-b948ea29ee33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:41.881011 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bf54b-efdc-4060-a5b0-b948ea29ee33 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:41.881338 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bf54b-efdc-4060-a5b0-b948ea29ee33 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:41.881612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bf54b-efdc-4060-a5b0-b948ea29ee33 None admin] PATH_INFO: `/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/8fc940594a1848aa9644b645049a1bd6/roles/f0fd911d7c9246aeb3d67299473a9f42` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:41.894965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74b74c-aa79-4013-af58-17648a6060a1 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:41.900478 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524bf54b-efdc-4060-a5b0-b948ea29ee33 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9c57a581324217ad2e64a0c9c7fdee, user_id=8fc940594a1848aa9644b645049a1bd6, role_id=f0fd911d7c9246aeb3d67299473a9f42)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:41.902264 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524bf54b-efdc-4060-a5b0-b948ea29ee33 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:41.929873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 414/834] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:31:41 2022] PUT /identity/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/8fc940594a1848aa9644b645049a1bd6/roles/f0fd911d7c9246aeb3d67299473a9f42 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:31:41.958831 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106e498-df70-4cf2-85b9-ccbcff1d63dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:41.960030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106e498-df70-4cf2-85b9-ccbcff1d63dc None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:41.960572 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106e498-df70-4cf2-85b9-ccbcff1d63dc None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:41.961063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106e498-df70-4cf2-85b9-ccbcff1d63dc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:41.962151 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f106e498-df70-4cf2-85b9-ccbcff1d63dc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:41.964942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f106e498-df70-4cf2-85b9-ccbcff1d63dc None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:41.982881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-f106e498-df70-4cf2-85b9-ccbcff1d63dc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:31:41.982881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:41.982881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:31:41.982881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:31:41.982881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:31:41.982881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:31:41.982881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:31:41.982881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:41.982881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:31:41.982881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:41.982881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:41.982881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:41.982881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:41.982881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:41.982881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:41.982881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:41.982881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:41.985251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:41.985251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:41.985251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:41.985251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:41.985251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:41.985251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:41.985251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:31:41.985251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:31:41.985251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:31:41.985251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:31:41.985251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:41.985251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:41.985251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:41.985251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:41.985251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:41.985251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:41.986959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:41.986959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:41.986959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:41.986959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:41.986959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:41.986959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:41.986959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:41.986959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:41.986959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:41.986959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:41.986959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:31:41.986959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:41.986959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:31:41.986959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:41.986959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 421/835] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:31:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:31:41.993208 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bf54b-efdc-4060-a5b0-b948ea29ee33 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:42.131310 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:42.136676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bf54b-efdc-4060-a5b0-b948ea29ee33 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:42.150466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf404a27-c8d2-45ed-8711-bb9ac9cdc6f3 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:42.151014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf404a27-c8d2-45ed-8711-bb9ac9cdc6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:42.153399 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf404a27-c8d2-45ed-8711-bb9ac9cdc6f3 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:42.156697 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 422/836] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:42.163891 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ecf64-0549-479e-b00a-c135613f7db8 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:42.164411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ecf64-0549-479e-b00a-c135613f7db8 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:42.164906 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ecf64-0549-479e-b00a-c135613f7db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:42.184589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805d9cd-acd5-462e-9105-9d5cd68c9181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:42.185223 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805d9cd-acd5-462e-9105-9d5cd68c9181 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:42.185498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805d9cd-acd5-462e-9105-9d5cd68c9181 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:42.186404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805d9cd-acd5-462e-9105-9d5cd68c9181 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:42.187211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805d9cd-acd5-462e-9105-9d5cd68c9181 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:42.187516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805d9cd-acd5-462e-9105-9d5cd68c9181 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:42.188955 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805d9cd-acd5-462e-9105-9d5cd68c9181 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:42.194693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 415/837] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:31:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:42.199838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-405ecf64-0549-479e-b00a-c135613f7db8 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:42.200839 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805d9cd-acd5-462e-9105-9d5cd68c9181 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:42.202352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-405ecf64-0549-479e-b00a-c135613f7db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:42.302230 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 423/838] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:42.408624 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805d9cd-acd5-462e-9105-9d5cd68c9181 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:42.474674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056343e9-3129-40bc-8d0a-9f947da073ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:42.476083 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056343e9-3129-40bc-8d0a-9f947da073ec None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:42.476782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056343e9-3129-40bc-8d0a-9f947da073ec None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:42.477455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056343e9-3129-40bc-8d0a-9f947da073ec None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:42.478759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056343e9-3129-40bc-8d0a-9f947da073ec None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:42.479498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056343e9-3129-40bc-8d0a-9f947da073ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:42.482346 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056343e9-3129-40bc-8d0a-9f947da073ec None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:42.493006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 416/839] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:31:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:31:42.654259 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ecf64-0549-479e-b00a-c135613f7db8 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:42.702496 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:42.800828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ecf64-0549-479e-b00a-c135613f7db8 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:42.837216 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632cb5f-658f-4eac-b1ec-38383631739b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:42.837883 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632cb5f-658f-4eac-b1ec-38383631739b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:42.838161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632cb5f-658f-4eac-b1ec-38383631739b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:42.838429 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632cb5f-658f-4eac-b1ec-38383631739b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:42.839039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0632cb5f-658f-4eac-b1ec-38383631739b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:42.840512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0632cb5f-658f-4eac-b1ec-38383631739b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:42.850955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-0632cb5f-658f-4eac-b1ec-38383631739b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:31:42.850955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:42.850955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:31:42.850955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:31:42.850955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:31:42.850955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:31:42.850955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:31:42.850955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:42.850955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:31:42.850955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:42.850955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:42.850955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:42.850955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:42.850955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:42.850955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:42.850955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:42.852030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:42.852030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:42.852030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:42.852030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:42.852030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:42.852030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:42.852030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:42.852030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:31:42.852030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:31:42.852030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:31:42.852030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:31:42.852030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:42.852030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:42.852030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:42.852030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:42.852030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:42.852030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:42.852949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:42.852949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:42.852949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:42.852949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:42.852949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:42.852949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:42.852949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:42.852949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:42.852949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:42.852949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:42.852949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:31:42.852949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:42.852949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:31:42.852949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:42.852949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 424/840] 10.222.0.25 () {56 vars in 1132 bytes} [Tue Aug 30 14:31:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:42.860631 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056343e9-3129-40bc-8d0a-9f947da073ec None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:42.896500 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:42.981700 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056343e9-3129-40bc-8d0a-9f947da073ec None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:43.023416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18326e96-2bdc-4c8a-a4ab-1b1b6b6000c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:43.024498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18326e96-2bdc-4c8a-a4ab-1b1b6b6000c3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:43.025026 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18326e96-2bdc-4c8a-a4ab-1b1b6b6000c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:43.025598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18326e96-2bdc-4c8a-a4ab-1b1b6b6000c3 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:43.026804 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18326e96-2bdc-4c8a-a4ab-1b1b6b6000c3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:43.027590 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18326e96-2bdc-4c8a-a4ab-1b1b6b6000c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:43.029267 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18326e96-2bdc-4c8a-a4ab-1b1b6b6000c3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:43.035692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 417/841] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Aug 30 14:31:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:43.046173 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632cb5f-658f-4eac-b1ec-38383631739b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:43.173219 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:43.186958 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632cb5f-658f-4eac-b1ec-38383631739b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:43.232411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f240b-da77-47e6-9609-2ecb217694e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:43.233314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f240b-da77-47e6-9609-2ecb217694e8 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:43.233779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f240b-da77-47e6-9609-2ecb217694e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:43.234114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f240b-da77-47e6-9609-2ecb217694e8 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:43.234780 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f240b-da77-47e6-9609-2ecb217694e8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:43.235082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f240b-da77-47e6-9609-2ecb217694e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:43.236522 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f240b-da77-47e6-9609-2ecb217694e8 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:43.242916 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 425/842] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Aug 30 14:31:43 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:43.576916 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63764660-c861-43c3-9c53-d21c2df5c525 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:43.577463 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63764660-c861-43c3-9c53-d21c2df5c525 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:43.577463 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63764660-c861-43c3-9c53-d21c2df5c525 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:43.579126 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 418/843] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:43.588286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2aea8a-d752-4c7e-a819-7b20068d4741 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:43.588632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2aea8a-d752-4c7e-a819-7b20068d4741 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:43.588836 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2aea8a-d752-4c7e-a819-7b20068d4741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:43.609717 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-ff2aea8a-d752-4c7e-a819-7b20068d4741 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:43.610955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-ff2aea8a-d752-4c7e-a819-7b20068d4741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:43.697183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 426/844] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:31:43.974491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63764660-c861-43c3-9c53-d21c2df5c525 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:44.078373 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:44.089104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63764660-c861-43c3-9c53-d21c2df5c525 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:44.126007 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe71786-9484-41d3-9281-84f65bb022d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:44.126660 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe71786-9484-41d3-9281-84f65bb022d6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:44.126935 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe71786-9484-41d3-9281-84f65bb022d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:44.127203 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe71786-9484-41d3-9281-84f65bb022d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:44.138648 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe71786-9484-41d3-9281-84f65bb022d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:44.143739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-9fe71786-9484-41d3-9281-84f65bb022d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:44.143739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:44.143739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:44.143739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:44.143739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:44.143739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:44.143739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:44.143739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:44.143739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:44.143739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:44.143739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:44.143739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:44.143739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:44.143739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:44.143739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:44.143739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:44.144771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:44.144771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:44.144771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:44.144771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:44.144771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:44.144771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:44.144771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:44.144771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:44.144771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:44.144771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:44.144771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:44.144771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:44.144771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:44.144771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:44.144771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:44.144771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:44.145627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:44.145627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:44.145627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:44.145627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:44.145627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:44.145627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:44.145627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:44.145627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:44.145627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:44.145627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:44.145627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:44.145627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:44.145627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 419/845] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:44.153126 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2aea8a-d752-4c7e-a819-7b20068d4741 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:44.250904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:44.267944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2aea8a-d752-4c7e-a819-7b20068d4741 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:44.305115 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0698be-69a4-43ee-a16a-17700d9dd2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:44.305738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0698be-69a4-43ee-a16a-17700d9dd2cd None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:44.306015 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0698be-69a4-43ee-a16a-17700d9dd2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:44.306285 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0698be-69a4-43ee-a16a-17700d9dd2cd None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:44.306973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0698be-69a4-43ee-a16a-17700d9dd2cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:44.307281 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0698be-69a4-43ee-a16a-17700d9dd2cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:44.308703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0698be-69a4-43ee-a16a-17700d9dd2cd None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:44.318780 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 427/846] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:31:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:44.325269 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe71786-9484-41d3-9281-84f65bb022d6 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:44.461341 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe71786-9484-41d3-9281-84f65bb022d6 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:44.529624 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359e05d-7f04-42e6-9d67-e3c047e3705a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:44.530645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359e05d-7f04-42e6-9d67-e3c047e3705a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:44.531109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359e05d-7f04-42e6-9d67-e3c047e3705a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:44.531570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359e05d-7f04-42e6-9d67-e3c047e3705a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:44.550193 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359e05d-7f04-42e6-9d67-e3c047e3705a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:44.552396 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359e05d-7f04-42e6-9d67-e3c047e3705a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:44.569449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-4359e05d-7f04-42e6-9d67-e3c047e3705a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:44.569449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:44.569449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:44.569449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:44.569449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:44.569449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:44.569449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:44.569449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:44.569449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:44.569449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:44.569449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:44.569449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:44.569449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:44.569449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:44.569449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:44.569449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:44.570504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:44.570504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:44.570504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:44.570504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:44.570504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:44.570504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:44.570504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:44.570504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:44.570504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:44.570504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:44.570504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:44.570504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:44.570504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:44.570504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:44.570504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:44.570504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:44.570504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:44.571423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:44.571423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:44.571423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:44.571423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 420/847] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:44.581101 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0698be-69a4-43ee-a16a-17700d9dd2cd None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:44.621798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f212e-3394-4bb6-bb8f-1407db6de384 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:44.622370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f212e-3394-4bb6-bb8f-1407db6de384 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:44.622370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f212e-3394-4bb6-bb8f-1407db6de384 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:44.623757 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 421/848] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:31:44.632128 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d4160-fa2b-4097-9d07-d106f417e864 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:44.632408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d4160-fa2b-4097-9d07-d106f417e864 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:44.632679 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d4160-fa2b-4097-9d07-d106f417e864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:44.652403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:44.656198 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-734d4160-fa2b-4097-9d07-d106f417e864 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:44.657587 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-734d4160-fa2b-4097-9d07-d106f417e864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:44.705057 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0698be-69a4-43ee-a16a-17700d9dd2cd None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:44.739541 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 422/849] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:31:44.740719 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0798b27-8145-43c5-a32e-b75891c18e0a None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:44.741005 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0798b27-8145-43c5-a32e-b75891c18e0a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:44.741288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0798b27-8145-43c5-a32e-b75891c18e0a None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:44.742496 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfa95d-0604-4c72-8e3a-d6bb927f53e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:44.746617 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 423/850] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:44.746705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfa95d-0604-4c72-8e3a-d6bb927f53e6 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:44.746705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfa95d-0604-4c72-8e3a-d6bb927f53e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:44.746705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfa95d-0604-4c72-8e3a-d6bb927f53e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:44.746705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dfa95d-0604-4c72-8e3a-d6bb927f53e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:44.746705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dfa95d-0604-4c72-8e3a-d6bb927f53e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:44.747570 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dfa95d-0604-4c72-8e3a-d6bb927f53e6 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:44.749758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a4706-f456-47ec-a353-598abcbf9e95 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:44.750917 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a4706-f456-47ec-a353-598abcbf9e95 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:44.751418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a4706-f456-47ec-a353-598abcbf9e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:44.761583 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 428/851] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:44.769117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfa95d-0604-4c72-8e3a-d6bb927f53e6 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:44.774608 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a56a4706-f456-47ec-a353-598abcbf9e95 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:44.776031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a56a4706-f456-47ec-a353-598abcbf9e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:44.778690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:44.876337 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 424/852] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:44.901238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfa95d-0604-4c72-8e3a-d6bb927f53e6 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:44.941236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54249f-0fe1-4e81-8205-8138878d5169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:44.941888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54249f-0fe1-4e81-8205-8138878d5169 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:44.942165 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54249f-0fe1-4e81-8205-8138878d5169 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:44.942437 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54249f-0fe1-4e81-8205-8138878d5169 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:44.951078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca54249f-0fe1-4e81-8205-8138878d5169 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:44.953042 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca54249f-0fe1-4e81-8205-8138878d5169 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:44.967511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-ca54249f-0fe1-4e81-8205-8138878d5169 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:31:44.967511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:44.967511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:44.967511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:44.967511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:44.967511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:44.967511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:44.967511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:44.967511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:44.967511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:44.967511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:44.967511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:44.967511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:44.967511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:44.967511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:44.967511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:44.969144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:44.970720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:44.970720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:44.970720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:44.970720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:44.970720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:44.970720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:44.970720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:44.970720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:44.970720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:31:44.970720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:44.970720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 429/853] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:44.975491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a4706-f456-47ec-a353-598abcbf9e95 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:45.036581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54249f-0fe1-4e81-8205-8138878d5169 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:45.051494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:45.103488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a4706-f456-47ec-a353-598abcbf9e95 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:45.124529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:45.144104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35951815-ad5a-4662-bd47-83c4911185ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:45.144648 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35951815-ad5a-4662-bd47-83c4911185ff None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:45.144943 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35951815-ad5a-4662-bd47-83c4911185ff None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:45.145238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35951815-ad5a-4662-bd47-83c4911185ff None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:45.146090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35951815-ad5a-4662-bd47-83c4911185ff None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:45.146443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35951815-ad5a-4662-bd47-83c4911185ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:45.148179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35951815-ad5a-4662-bd47-83c4911185ff None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:45.174507 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 425/854] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Aug 30 14:31:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:45.174997 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54249f-0fe1-4e81-8205-8138878d5169 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:45.175591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35951815-ad5a-4662-bd47-83c4911185ff None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:45.216092 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d144c0-c81d-453c-bbbe-17cf6d2bfc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:45.216738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d144c0-c81d-453c-bbbe-17cf6d2bfc06 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:45.217164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d144c0-c81d-453c-bbbe-17cf6d2bfc06 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:45.217430 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d144c0-c81d-453c-bbbe-17cf6d2bfc06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:45.229497 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d144c0-c81d-453c-bbbe-17cf6d2bfc06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:45.235020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-c0d144c0-c81d-453c-bbbe-17cf6d2bfc06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:45.235020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:45.235020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:45.235020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:45.235020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:45.235020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:45.235020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:45.235020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:45.235020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:45.235020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:45.235020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:45.235020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:45.235020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:45.235020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:45.235020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:45.235020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:45.236210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:45.236210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:45.236210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:45.236210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:45.236210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:45.236210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:45.236210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:45.236210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:45.236210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:45.236210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:45.236210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:45.236210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:45.236210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:45.236210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:45.236210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:45.236210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:45.237129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:45.237129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:45.237129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:45.237129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:45.237129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:45.237129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:45.237129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:45.237129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:45.237129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:45.237129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:45.237129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:31:45.237129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:45.237129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 430/855] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:31:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:31:45.237129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d144c0-c81d-453c-bbbe-17cf6d2bfc06 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:45.293618 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35951815-ad5a-4662-bd47-83c4911185ff None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:45.319941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:45.335742 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59003471-a415-4b7f-a8fc-6359369cf0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:45.336466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59003471-a415-4b7f-a8fc-6359369cf0f7 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:45.336751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59003471-a415-4b7f-a8fc-6359369cf0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:45.337027 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59003471-a415-4b7f-a8fc-6359369cf0f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:45.362421 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d144c0-c81d-453c-bbbe-17cf6d2bfc06 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:45.401395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:45.418010 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88fae3-b4bd-4c83-a095-c89dfac4c0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:45.418758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88fae3-b4bd-4c83-a095-c89dfac4c0b2 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:45.419035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88fae3-b4bd-4c83-a095-c89dfac4c0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:45.419301 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88fae3-b4bd-4c83-a095-c89dfac4c0b2 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:45.420246 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88fae3-b4bd-4c83-a095-c89dfac4c0b2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:45.422370 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88fae3-b4bd-4c83-a095-c89dfac4c0b2 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:45.445263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 431/856] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Aug 30 14:31:45 2022] POST /identity/v3/services => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:31:45.446620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59003471-a415-4b7f-a8fc-6359369cf0f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:45.446824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88fae3-b4bd-4c83-a095-c89dfac4c0b2 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:45.450011 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59003471-a415-4b7f-a8fc-6359369cf0f7 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:45.579585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88fae3-b4bd-4c83-a095-c89dfac4c0b2 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:45.619647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 426/857] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:45 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:31:45.621722 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59003471-a415-4b7f-a8fc-6359369cf0f7 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:45.635405 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88d95b-a4ce-4418-b33d-9471272b7a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:45.636070 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88d95b-a4ce-4418-b33d-9471272b7a93 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:45.636351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88d95b-a4ce-4418-b33d-9471272b7a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:45.636635 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88d95b-a4ce-4418-b33d-9471272b7a93 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:45.637300 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88d95b-a4ce-4418-b33d-9471272b7a93 None admin] RBAC: Adding query filter params (name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:45.637601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88d95b-a4ce-4418-b33d-9471272b7a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:45.639060 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88d95b-a4ce-4418-b33d-9471272b7a93 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:45.649440 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 432/858] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:31:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:45.656340 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88d95b-a4ce-4418-b33d-9471272b7a93 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:45.661176 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:45.721571 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:45.764853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59003471-a415-4b7f-a8fc-6359369cf0f7 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:45.813857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88d95b-a4ce-4418-b33d-9471272b7a93 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:45.814290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c471a6-8e09-4528-80ac-01721c2958b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:45.815086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c471a6-8e09-4528-80ac-01721c2958b8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:45.815464 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c471a6-8e09-4528-80ac-01721c2958b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:45.815837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c471a6-8e09-4528-80ac-01721c2958b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:45.834300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c471a6-8e09-4528-80ac-01721c2958b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:45.836529 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c471a6-8e09-4528-80ac-01721c2958b8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:45.859923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-42c471a6-8e09-4528-80ac-01721c2958b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:45.859923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:45.859923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:45.859923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:45.859923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:45.859923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:45.859923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:45.859923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:45.859923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:45.859923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:45.859923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:45.859923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:45.859923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:45.859923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:45.859923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:45.859923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:45.861389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:45.861389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:45.861389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:45.861389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:45.861389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:45.861389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:45.861389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:45.861389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:45.861389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:45.861389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:45.861389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:45.861389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:45.861389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:45.861389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:45.861389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:45.861389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:45.861389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:45.862598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:45.862598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:45.862598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:45.862916 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 427/859] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:45.869754 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c471a6-8e09-4528-80ac-01721c2958b8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:45.870896 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52492aeb-e371-43a9-a492-2c0ce9a8c437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:45.871564 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52492aeb-e371-43a9-a492-2c0ce9a8c437 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:45.871852 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52492aeb-e371-43a9-a492-2c0ce9a8c437 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:45.872152 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52492aeb-e371-43a9-a492-2c0ce9a8c437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:45.890923 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52492aeb-e371-43a9-a492-2c0ce9a8c437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:45.892452 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52492aeb-e371-43a9-a492-2c0ce9a8c437 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:45.915114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-52492aeb-e371-43a9-a492-2c0ce9a8c437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:45.915114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:45.915114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:45.915114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:45.915114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:45.915114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:45.915114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:45.915114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:45.915114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:45.915114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:45.915114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:45.915114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:45.915114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:45.915114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:45.915114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:45.915114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:45.916146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:45.916146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:45.916146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:45.916146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:45.916146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:45.916146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:45.916146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:45.916146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:45.916146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:45.916146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:45.916146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:45.916146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:45.916146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:45.916146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:45.916146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:45.916146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:45.916146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:45.917066 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:45.917066 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:45.917066 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:45.917996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 433/860] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:45.922772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52492aeb-e371-43a9-a492-2c0ce9a8c437 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:45.981435 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:46.027126 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c471a6-8e09-4528-80ac-01721c2958b8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:46.077068 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:46.085132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52492aeb-e371-43a9-a492-2c0ce9a8c437 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:46.087253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af534c3-d62d-446a-a922-b84d9be5ae81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:46.087887 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af534c3-d62d-446a-a922-b84d9be5ae81 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:46.088204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af534c3-d62d-446a-a922-b84d9be5ae81 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:46.088453 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af534c3-d62d-446a-a922-b84d9be5ae81 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:46.089279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af534c3-d62d-446a-a922-b84d9be5ae81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:46.089576 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af534c3-d62d-446a-a922-b84d9be5ae81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:46.091212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af534c3-d62d-446a-a922-b84d9be5ae81 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:46.107003 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 428/861] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:46.108737 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f828f86-4d1f-4ec4-a655-befca8b56a40 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:46.109110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f828f86-4d1f-4ec4-a655-befca8b56a40 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:46.109358 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f828f86-4d1f-4ec4-a655-befca8b56a40 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:46.111068 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 429/862] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:46.115987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f828f86-4d1f-4ec4-a655-befca8b56a40 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:46.145957 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b094f-ed8a-4c03-8ad0-fc9c13672f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:46.146599 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b094f-ed8a-4c03-8ad0-fc9c13672f81 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:46.146872 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b094f-ed8a-4c03-8ad0-fc9c13672f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:46.147139 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b094f-ed8a-4c03-8ad0-fc9c13672f81 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:46.147949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b094f-ed8a-4c03-8ad0-fc9c13672f81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:46.148301 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b094f-ed8a-4c03-8ad0-fc9c13672f81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:46.149717 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b094f-ed8a-4c03-8ad0-fc9c13672f81 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:46.172754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 434/863] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:31:46.172754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb9786-6235-4007-8e57-0d697fb65814 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:46.172754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb9786-6235-4007-8e57-0d697fb65814 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:46.172754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb9786-6235-4007-8e57-0d697fb65814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:46.191670 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-08eb9786-6235-4007-8e57-0d697fb65814 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:46.192932 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-08eb9786-6235-4007-8e57-0d697fb65814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:46.271915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 435/864] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:46 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:46.273134 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb9786-6235-4007-8e57-0d697fb65814 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:46.298810 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f828f86-4d1f-4ec4-a655-befca8b56a40 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:46.346039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0322c4-86d3-405c-bd70-5c161ba64eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:46.347793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0322c4-86d3-405c-bd70-5c161ba64eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:46.348238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0322c4-86d3-405c-bd70-5c161ba64eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:46.348744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0322c4-86d3-405c-bd70-5c161ba64eb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:46.351538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:46.360507 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0322c4-86d3-405c-bd70-5c161ba64eb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:46.362507 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0322c4-86d3-405c-bd70-5c161ba64eb3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:46.377764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-ed0322c4-86d3-405c-bd70-5c161ba64eb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:46.377764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:46.377764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:46.377764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:46.377764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:46.377764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:46.377764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:46.377764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:46.377764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:46.377764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:46.377764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:46.377764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:46.377764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:46.377764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:46.377764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:46.378755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:46.378755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:46.378755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:46.378755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:46.378755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:46.378755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:46.378755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:46.378755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:46.378755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:46.378755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:46.378755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:46.378755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:46.378755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:46.378755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:46.378755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:46.378755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:46.378755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:46.379647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:46.379647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:46.379647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:46.379647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:46.379647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:46.379647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:46.379647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:46.379647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:46.387352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 430/865] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:46.390331 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0322c4-86d3-405c-bd70-5c161ba64eb3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:46.438531 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:46.459362 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb9786-6235-4007-8e57-0d697fb65814 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:46.506254 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef41821-0973-4314-b4b3-57a8494f929b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:46.507672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef41821-0973-4314-b4b3-57a8494f929b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:46.508659 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef41821-0973-4314-b4b3-57a8494f929b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:46.509337 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef41821-0973-4314-b4b3-57a8494f929b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:46.523596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef41821-0973-4314-b4b3-57a8494f929b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:46.526381 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef41821-0973-4314-b4b3-57a8494f929b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:46.546701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0322c4-86d3-405c-bd70-5c161ba64eb3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:46.550411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-7ef41821-0973-4314-b4b3-57a8494f929b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:31:46.550411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:46.550411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:46.550411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:46.550411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:46.550411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:46.550411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:46.550411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:46.550411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:46.550411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:46.550411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:46.550411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:46.550411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:46.550411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:46.550411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:46.550411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:46.551352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:46.552308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:46.552308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:46.552308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:46.552308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:46.552308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:46.552308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:46.552308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:46.552308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:46.552308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:31:46.552308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:46.558516 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 436/866] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Aug 30 14:31:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:46.558981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef41821-0973-4314-b4b3-57a8494f929b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:46.595789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c17f2d-44c0-42c2-b1d6-1cde78d7248a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:46.596466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c17f2d-44c0-42c2-b1d6-1cde78d7248a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:46.596752 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c17f2d-44c0-42c2-b1d6-1cde78d7248a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:46.597025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c17f2d-44c0-42c2-b1d6-1cde78d7248a None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:46.597735 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c17f2d-44c0-42c2-b1d6-1cde78d7248a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:46.598048 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c17f2d-44c0-42c2-b1d6-1cde78d7248a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:46.599607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c17f2d-44c0-42c2-b1d6-1cde78d7248a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:46.620079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 431/867] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:46.620079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c17f2d-44c0-42c2-b1d6-1cde78d7248a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:46.685835 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:46.703234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef41821-0973-4314-b4b3-57a8494f929b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:46.741244 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c17f2d-44c0-42c2-b1d6-1cde78d7248a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:46.753386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff8b02-70b8-449a-93ef-2723570147d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:46.754348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff8b02-70b8-449a-93ef-2723570147d4 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:46.755068 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff8b02-70b8-449a-93ef-2723570147d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:46.755545 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff8b02-70b8-449a-93ef-2723570147d4 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:46.756466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff8b02-70b8-449a-93ef-2723570147d4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:46.758298 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff8b02-70b8-449a-93ef-2723570147d4 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:46.788428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 437/868] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Aug 30 14:31:46 2022] POST /identity/v3/services => generated 269 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:31:46.789358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff8b02-70b8-449a-93ef-2723570147d4 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:46.791598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19d4c0-0398-495a-8204-b601c3a147bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:46.792486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19d4c0-0398-495a-8204-b601c3a147bd None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:46.792873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19d4c0-0398-495a-8204-b601c3a147bd None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:46.793239 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19d4c0-0398-495a-8204-b601c3a147bd None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:46.794397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19d4c0-0398-495a-8204-b601c3a147bd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:46.795268 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19d4c0-0398-495a-8204-b601c3a147bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:46.797177 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19d4c0-0398-495a-8204-b601c3a147bd None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:46.810933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:46.847183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 432/869] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Aug 30 14:31:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:46.853200 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19d4c0-0398-495a-8204-b601c3a147bd None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:46.986702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff8b02-70b8-449a-93ef-2723570147d4 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:47.002583 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19d4c0-0398-495a-8204-b601c3a147bd None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:47.020089 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:47.049670 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21557867-a58c-44aa-8053-01fa510801c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:47.050300 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21557867-a58c-44aa-8053-01fa510801c9 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:47.050569 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21557867-a58c-44aa-8053-01fa510801c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:47.050833 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21557867-a58c-44aa-8053-01fa510801c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:47.051736 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21557867-a58c-44aa-8053-01fa510801c9 None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=d5d98f5adfaf410580481417e7ba1638) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:47.052067 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21557867-a58c-44aa-8053-01fa510801c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:47.053714 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21557867-a58c-44aa-8053-01fa510801c9 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:47.062447 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 438/870] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:46 2022] GET /identity/v3/role_assignments?user.id=d5d98f5adfaf410580481417e7ba1638&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:47.071980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d0116-259a-4f81-8c0b-24d05d9f58d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:47.073142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d0116-259a-4f81-8c0b-24d05d9f58d1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:47.073610 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d0116-259a-4f81-8c0b-24d05d9f58d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:47.074101 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d0116-259a-4f81-8c0b-24d05d9f58d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:47.226198 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d0116-259a-4f81-8c0b-24d05d9f58d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:47.229071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d0116-259a-4f81-8c0b-24d05d9f58d1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:47.413187 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 433/871] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:46 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:31:47.422022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21557867-a58c-44aa-8053-01fa510801c9 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:47.557340 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:47.617742 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21557867-a58c-44aa-8053-01fa510801c9 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:47.682339 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd47b7-89a2-4d1d-82cf-d0d67f5f77fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:47.683314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd47b7-89a2-4d1d-82cf-d0d67f5f77fc None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:47.683783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd47b7-89a2-4d1d-82cf-d0d67f5f77fc None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:47.684269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd47b7-89a2-4d1d-82cf-d0d67f5f77fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:47.704893 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd47b7-89a2-4d1d-82cf-d0d67f5f77fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:47.707871 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd47b7-89a2-4d1d-82cf-d0d67f5f77fc None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:47.735657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-81fd47b7-89a2-4d1d-82cf-d0d67f5f77fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:47.735657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:47.735657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:47.735657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:47.735657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:47.735657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:47.735657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:47.735657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:47.735657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:47.735657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:47.735657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:47.735657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:47.735657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:47.735657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:47.735657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:47.735657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:47.736712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:47.736712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:47.736712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:47.736712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:47.736712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:47.736712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:47.736712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:47.736712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:47.736712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:47.736712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:47.736712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:47.736712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:47.736712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:47.736712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:47.736712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:47.736712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:47.736712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:47.737649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:47.737649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:47.737649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:47.738267 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 439/872] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:47.746401 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d0116-259a-4f81-8c0b-24d05d9f58d1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:47.768384 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:47.799193 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642c839-1098-48c0-a84b-6d6b52b2f78a None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:47.800870 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642c839-1098-48c0-a84b-6d6b52b2f78a None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:47.801579 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642c839-1098-48c0-a84b-6d6b52b2f78a None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:47.804948 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 440/873] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:47.814435 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1741448-5d97-4c24-95a7-03d898bce6a7 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:47.815442 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1741448-5d97-4c24-95a7-03d898bce6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:47.816264 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1741448-5d97-4c24-95a7-03d898bce6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:47.852208 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-c1741448-5d97-4c24-95a7-03d898bce6a7 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:47.854954 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c1741448-5d97-4c24-95a7-03d898bce6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:47.877834 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d0116-259a-4f81-8c0b-24d05d9f58d1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:47.915687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f505d2-2220-475c-a302-18e47d581437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:47.916343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f505d2-2220-475c-a302-18e47d581437 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:47.916616 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f505d2-2220-475c-a302-18e47d581437 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:47.916884 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f505d2-2220-475c-a302-18e47d581437 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:47.917545 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f505d2-2220-475c-a302-18e47d581437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:47.917843 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f505d2-2220-475c-a302-18e47d581437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:47.919350 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f505d2-2220-475c-a302-18e47d581437 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:47.939851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 434/874] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:47.941591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f505d2-2220-475c-a302-18e47d581437 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:47.980047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 441/875] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:31:48.068449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:48.122648 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f505d2-2220-475c-a302-18e47d581437 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:48.167814 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736dfef-49b2-475b-94ac-3641eceb272a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:48.168582 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736dfef-49b2-475b-94ac-3641eceb272a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:48.168861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736dfef-49b2-475b-94ac-3641eceb272a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:48.169163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736dfef-49b2-475b-94ac-3641eceb272a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:48.178527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5736dfef-49b2-475b-94ac-3641eceb272a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:48.180087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5736dfef-49b2-475b-94ac-3641eceb272a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:48.196615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-5736dfef-49b2-475b-94ac-3641eceb272a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:48.196615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:48.196615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:48.196615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:48.196615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:48.196615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:48.196615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:48.196615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:48.196615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:48.196615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:48.196615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:48.196615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:48.196615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:48.196615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:48.196615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:48.198017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:48.198017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:48.198017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:48.198017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:48.198017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:48.198017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:48.198017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:48.198017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:48.198017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:48.198017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:48.198017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:48.198017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:48.198017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:48.198017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:48.198017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:48.198017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:48.198017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:48.199120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:48.199120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:48.199120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:48.199120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:48.199120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:48.199120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:48.199120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:48.199120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:48.199120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 435/876] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:48.208059 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1741448-5d97-4c24-95a7-03d898bce6a7 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:48.270717 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:48.288996 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736dfef-49b2-475b-94ac-3641eceb272a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:48.341761 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1741448-5d97-4c24-95a7-03d898bce6a7 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:48.395013 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bb628-c204-45bb-9fc1-acdcb0cd978c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:48.395664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bb628-c204-45bb-9fc1-acdcb0cd978c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:48.396147 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bb628-c204-45bb-9fc1-acdcb0cd978c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:48.396421 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bb628-c204-45bb-9fc1-acdcb0cd978c None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:48.398376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bb628-c204-45bb-9fc1-acdcb0cd978c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:48.398700 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bb628-c204-45bb-9fc1-acdcb0cd978c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:48.400705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bb628-c204-45bb-9fc1-acdcb0cd978c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:48.420100 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 442/877] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:48.425821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bb628-c204-45bb-9fc1-acdcb0cd978c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:48.450144 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736dfef-49b2-475b-94ac-3641eceb272a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:48.485569 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:48.504981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61adfc4-462a-472d-be12-eef9531ed005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:48.505716 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61adfc4-462a-472d-be12-eef9531ed005 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:48.506001 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61adfc4-462a-472d-be12-eef9531ed005 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:48.506275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61adfc4-462a-472d-be12-eef9531ed005 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:48.506878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61adfc4-462a-472d-be12-eef9531ed005 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:48.508483 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61adfc4-462a-472d-be12-eef9531ed005 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:48.519016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-a61adfc4-462a-472d-be12-eef9531ed005 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:31:48.519016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:48.519016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:31:48.519016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:31:48.519016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:31:48.519016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:31:48.519016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:31:48.519016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:48.519016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:31:48.519016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:48.519016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:48.519016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:48.519016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:48.519016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:48.519016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:48.519016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:48.519016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:48.520439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:48.520439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:48.520439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:48.520439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:48.520439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:48.520439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:48.520439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:31:48.520439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:31:48.520439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:31:48.520439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:31:48.520439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:48.520439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:48.520439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:48.520439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:48.520439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:48.520439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:48.521336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:48.521336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:48.521336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:48.521336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:48.521336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:48.521336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:48.521336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:48.521336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:48.521336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:48.521336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:48.521336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:31:48.521336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:48.521336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:31:48.521336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:48.525566 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 436/878] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:31:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:31:48.526997 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61adfc4-462a-472d-be12-eef9531ed005 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:48.583937 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bb628-c204-45bb-9fc1-acdcb0cd978c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:48.602565 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:48.633749 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d976a3-9542-4ed4-8224-5864951ae235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:48.634893 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d976a3-9542-4ed4-8224-5864951ae235 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:48.635435 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d976a3-9542-4ed4-8224-5864951ae235 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:48.635932 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d976a3-9542-4ed4-8224-5864951ae235 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:48.637067 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d976a3-9542-4ed4-8224-5864951ae235 None admin] RBAC: Adding query filter params (role.id=f0fd911d7c9246aeb3d67299473a9f42, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=8fc940594a1848aa9644b645049a1bd6) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:48.637714 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d976a3-9542-4ed4-8224-5864951ae235 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:48.639847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d976a3-9542-4ed4-8224-5864951ae235 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:48.656357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 443/879] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:48 2022] GET /identity/v3/role_assignments?user.id=8fc940594a1848aa9644b645049a1bd6&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=f0fd911d7c9246aeb3d67299473a9f42 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:31:48.686758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61adfc4-462a-472d-be12-eef9531ed005 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:48.737078 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91210409-2ed4-41c6-b07f-92a2ba0349d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:48.737615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91210409-2ed4-41c6-b07f-92a2ba0349d3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:48.737965 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91210409-2ed4-41c6-b07f-92a2ba0349d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:48.738241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91210409-2ed4-41c6-b07f-92a2ba0349d3 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:48.738900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91210409-2ed4-41c6-b07f-92a2ba0349d3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:48.739203 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91210409-2ed4-41c6-b07f-92a2ba0349d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:48.740659 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91210409-2ed4-41c6-b07f-92a2ba0349d3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:48.747377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 437/880] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:31:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:48.754366 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d976a3-9542-4ed4-8224-5864951ae235 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:48.902129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d976a3-9542-4ed4-8224-5864951ae235 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:48.956145 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2944d-b2af-4bdf-a42e-68e2a3dc6bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:48.957961 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2944d-b2af-4bdf-a42e-68e2a3dc6bac None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:48.958705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2944d-b2af-4bdf-a42e-68e2a3dc6bac None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:48.959399 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2944d-b2af-4bdf-a42e-68e2a3dc6bac None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:48.960551 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2944d-b2af-4bdf-a42e-68e2a3dc6bac None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:48.961624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2944d-b2af-4bdf-a42e-68e2a3dc6bac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:48.964492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2944d-b2af-4bdf-a42e-68e2a3dc6bac None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:48.974257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 444/881] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:31:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:48.984075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91210409-2ed4-41c6-b07f-92a2ba0349d3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:49.070536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:49.089017 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4485ec-d287-444e-8cce-2d820aeb75bf None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:49.089341 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4485ec-d287-444e-8cce-2d820aeb75bf None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:49.089564 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4485ec-d287-444e-8cce-2d820aeb75bf None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:49.091211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 445/882] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:49.098409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae4611-55e3-43a6-aeac-aa352e6d190f None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:49.098999 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae4611-55e3-43a6-aeac-aa352e6d190f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:49.099531 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae4611-55e3-43a6-aeac-aa352e6d190f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:49.121519 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-c0ae4611-55e3-43a6-aeac-aa352e6d190f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:49.123103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c0ae4611-55e3-43a6-aeac-aa352e6d190f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:49.130379 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91210409-2ed4-41c6-b07f-92a2ba0349d3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:49.176128 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf5f53-f103-433d-b43c-a5329d65428e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:49.177160 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf5f53-f103-433d-b43c-a5329d65428e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:49.177730 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf5f53-f103-433d-b43c-a5329d65428e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:49.178248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf5f53-f103-433d-b43c-a5329d65428e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:49.179228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf5f53-f103-433d-b43c-a5329d65428e None admin] RBAC: Adding query filter params (interface=public, service_id=69771caf40ca4aebb0cd0b9c476efd0c, region_id=RegionOne) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:49.179719 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf5f53-f103-433d-b43c-a5329d65428e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:49.180708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:49.183739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf5f53-f103-433d-b43c-a5329d65428e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:49.197870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 438/883] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Aug 30 14:31:48 2022] GET /identity/v3/endpoints?service_id=69771caf40ca4aebb0cd0b9c476efd0c&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:49.199803 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf5f53-f103-433d-b43c-a5329d65428e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:49.236131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 446/884] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:31:49.342428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf5f53-f103-433d-b43c-a5329d65428e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:49.369483 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:49.382248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3998943-3265-47cc-8712-1a5083e4e239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:49.383173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3998943-3265-47cc-8712-1a5083e4e239 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:49.384027 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3998943-3265-47cc-8712-1a5083e4e239 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:49.384815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3998943-3265-47cc-8712-1a5083e4e239 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:49.385518 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3998943-3265-47cc-8712-1a5083e4e239 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:49.388240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3998943-3265-47cc-8712-1a5083e4e239 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:49.398866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 439/885] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Aug 30 14:31:49 2022] GET /identity/v3/services => generated 1268 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:31:49.603974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae4611-55e3-43a6-aeac-aa352e6d190f None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:49.728166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae4611-55e3-43a6-aeac-aa352e6d190f None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:49.773527 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9ac7d-ee44-4d7f-ae28-02e94d11ce1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:49.774768 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9ac7d-ee44-4d7f-ae28-02e94d11ce1f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:49.775305 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9ac7d-ee44-4d7f-ae28-02e94d11ce1f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:49.775820 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9ac7d-ee44-4d7f-ae28-02e94d11ce1f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:49.776768 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9ac7d-ee44-4d7f-ae28-02e94d11ce1f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:49.778838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9ac7d-ee44-4d7f-ae28-02e94d11ce1f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:49.790177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-60f9ac7d-ee44-4d7f-ae28-02e94d11ce1f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:31:49.790177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:49.790177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:31:49.790177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:31:49.790177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:31:49.790177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:31:49.790177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:31:49.790177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:49.790177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:31:49.790177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:49.790177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:49.790177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:49.790177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:49.790177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:49.790177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:49.790177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:49.791220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:49.791220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:49.791220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:49.791220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:49.791220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:49.791220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:49.791220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:49.791220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:31:49.791220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:31:49.791220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:31:49.791220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:31:49.791220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:49.791220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:49.791220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:49.791220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:49.791220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:49.791220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:49.792187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:49.792187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:49.792187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:49.792187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:49.792187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:49.792187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:49.792187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:49.792187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:49.792187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:49.792187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:49.792187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:31:49.792187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:49.792187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:31:49.792187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:49.795960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 447/886] 10.222.0.25 () {56 vars in 1132 bytes} [Tue Aug 30 14:31:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:49.805576 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3998943-3265-47cc-8712-1a5083e4e239 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:49.941699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3998943-3265-47cc-8712-1a5083e4e239 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:49.948930 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc0736-5535-4412-a997-fcc44da04196 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:49.949711 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc0736-5535-4412-a997-fcc44da04196 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:49.949711 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc0736-5535-4412-a997-fcc44da04196 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:49.951608 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 448/887] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:49.960301 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1620ef-0f59-4c69-9b9b-b1464358d887 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:49.960765 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1620ef-0f59-4c69-9b9b-b1464358d887 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:49.961235 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1620ef-0f59-4c69-9b9b-b1464358d887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:49.987320 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-9d1620ef-0f59-4c69-9b9b-b1464358d887 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:49.988596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-9d1620ef-0f59-4c69-9b9b-b1464358d887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:50.024338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:50.025041 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4c706-5a4d-4f92-82f0-82121d894014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:50.026035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4c706-5a4d-4f92-82f0-82121d894014 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:50.026670 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4c706-5a4d-4f92-82f0-82121d894014 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:50.026935 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4c706-5a4d-4f92-82f0-82121d894014 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:50.028796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4c706-5a4d-4f92-82f0-82121d894014 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:50.028796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4c706-5a4d-4f92-82f0-82121d894014 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:50.034072 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4c706-5a4d-4f92-82f0-82121d894014 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:50.041044 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 440/888] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Aug 30 14:31:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:50.046938 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4c706-5a4d-4f92-82f0-82121d894014 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:50.095679 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 449/889] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:50.113596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:50.179295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4c706-5a4d-4f92-82f0-82121d894014 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:50.222273 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb62ae-b708-4449-aeeb-beb34eb87ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:50.223191 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb62ae-b708-4449-aeeb-beb34eb87ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:50.223673 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb62ae-b708-4449-aeeb-beb34eb87ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:50.224129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb62ae-b708-4449-aeeb-beb34eb87ac8 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:50.225319 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb62ae-b708-4449-aeeb-beb34eb87ac8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:50.225784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb62ae-b708-4449-aeeb-beb34eb87ac8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:50.227620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb62ae-b708-4449-aeeb-beb34eb87ac8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:50.234911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 441/890] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Aug 30 14:31:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:31:50.243900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1620ef-0f59-4c69-9b9b-b1464358d887 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:50.358351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1620ef-0f59-4c69-9b9b-b1464358d887 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:50.399496 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52fdf1-75e6-4cd9-b1e2-28fefbf2acb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:50.400151 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52fdf1-75e6-4cd9-b1e2-28fefbf2acb4 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:50.400424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52fdf1-75e6-4cd9-b1e2-28fefbf2acb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:50.400684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52fdf1-75e6-4cd9-b1e2-28fefbf2acb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:50.401396 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52fdf1-75e6-4cd9-b1e2-28fefbf2acb4 None admin] RBAC: Adding query filter params (interface=public, service_id=ae12919cf6bc45c9a4eddd9713fd9999, region_id=RegionOne) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:50.401697 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52fdf1-75e6-4cd9-b1e2-28fefbf2acb4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:50.403058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52fdf1-75e6-4cd9-b1e2-28fefbf2acb4 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:50.409584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 450/891] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Aug 30 14:31:50 2022] GET /identity/v3/endpoints?service_id=ae12919cf6bc45c9a4eddd9713fd9999&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:50.417206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb62ae-b708-4449-aeeb-beb34eb87ac8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:50.563683 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb62ae-b708-4449-aeeb-beb34eb87ac8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:50.590364 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:50.590600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52fdf1-75e6-4cd9-b1e2-28fefbf2acb4 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:50.632762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7f126-95b0-46a6-9600-dcb8d658a950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:50.633911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7f126-95b0-46a6-9600-dcb8d658a950 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:50.634625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7f126-95b0-46a6-9600-dcb8d658a950 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:50.635167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7f126-95b0-46a6-9600-dcb8d658a950 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:50.636373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:50.637627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7f126-95b0-46a6-9600-dcb8d658a950 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:50.640582 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7f126-95b0-46a6-9600-dcb8d658a950 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:50.658022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 442/892] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Aug 30 14:31:50 2022] GET /identity/v3/services => generated 1268 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:31:50.747887 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52fdf1-75e6-4cd9-b1e2-28fefbf2acb4 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:50.787788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f1461-93bb-4c7e-87d3-15c78037a92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:50.788478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f1461-93bb-4c7e-87d3-15c78037a92c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:50.788760 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f1461-93bb-4c7e-87d3-15c78037a92c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:50.789021 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f1461-93bb-4c7e-87d3-15c78037a92c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:50.789617 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f1461-93bb-4c7e-87d3-15c78037a92c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:50.791029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f1461-93bb-4c7e-87d3-15c78037a92c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:50.800965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-567f1461-93bb-4c7e-87d3-15c78037a92c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:31:50.800965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:50.800965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:31:50.800965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:31:50.800965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:31:50.800965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:31:50.800965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:31:50.800965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:50.800965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:31:50.800965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:50.800965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:50.800965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:50.800965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:50.800965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:50.800965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:50.800965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:50.800965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:50.802063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:50.802063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:50.802063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:50.802063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:50.802063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:50.802063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:50.802063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:31:50.802063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:31:50.802063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:31:50.802063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:31:50.802063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:50.802063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:50.802063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:50.802063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:50.802063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:50.802063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:50.802924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:50.802924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:50.802924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:50.802924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:50.802924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:50.802924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:50.802924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:50.802924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:50.802924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:50.802924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:50.802924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:31:50.802924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:50.802924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:31:50.802924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:50.802924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 451/893] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Aug 30 14:31:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:50.812359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7f126-95b0-46a6-9600-dcb8d658a950 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:50.949024 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7f126-95b0-46a6-9600-dcb8d658a950 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:51.013359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03617689-a768-4053-87e5-57436777e68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:51.014432 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03617689-a768-4053-87e5-57436777e68c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:51.014910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03617689-a768-4053-87e5-57436777e68c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:51.015378 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03617689-a768-4053-87e5-57436777e68c None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:51.016554 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03617689-a768-4053-87e5-57436777e68c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:51.017070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03617689-a768-4053-87e5-57436777e68c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:51.019766 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03617689-a768-4053-87e5-57436777e68c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:51.022992 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae1dd4-f615-408f-8a4f-9ef9831ee2aa None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:51.023953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae1dd4-f615-408f-8a4f-9ef9831ee2aa None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:51.023953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae1dd4-f615-408f-8a4f-9ef9831ee2aa None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:51.025193 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 452/894] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:51.029380 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 443/895] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Aug 30 14:31:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:51.031303 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0f570-2c08-4e48-844f-f8a626775eed None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:51.031588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0f570-2c08-4e48-844f-f8a626775eed None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:51.031857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0f570-2c08-4e48-844f-f8a626775eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:51.040813 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03617689-a768-4053-87e5-57436777e68c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:51.057319 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-38b0f570-2c08-4e48-844f-f8a626775eed None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:51.058265 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-38b0f570-2c08-4e48-844f-f8a626775eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:51.097266 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:51.156227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 453/896] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:51 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:51.180282 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03617689-a768-4053-87e5-57436777e68c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:51.226725 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eeb383-fbd7-4160-930f-c302c6f6d403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:51.227259 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeb383-fbd7-4160-930f-c302c6f6d403 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:51.227536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeb383-fbd7-4160-930f-c302c6f6d403 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:51.227800 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeb383-fbd7-4160-930f-c302c6f6d403 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:51.228518 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eeb383-fbd7-4160-930f-c302c6f6d403 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:51.228815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eeb383-fbd7-4160-930f-c302c6f6d403 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:51.231253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eeb383-fbd7-4160-930f-c302c6f6d403 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:51.240808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 444/897] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Aug 30 14:31:51 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:51.420618 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0f570-2c08-4e48-844f-f8a626775eed None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:51.447568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:51.528958 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0f570-2c08-4e48-844f-f8a626775eed None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:51.566027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7e8dd-cb0a-483e-9b4a-0c4bc535184b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:51.566675 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7e8dd-cb0a-483e-9b4a-0c4bc535184b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:51.566953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7e8dd-cb0a-483e-9b4a-0c4bc535184b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:51.567217 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7e8dd-cb0a-483e-9b4a-0c4bc535184b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:51.579209 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7e8dd-cb0a-483e-9b4a-0c4bc535184b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:51.584467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-d4f7e8dd-cb0a-483e-9b4a-0c4bc535184b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:31:51.584467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:51.584467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:51.584467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:51.584467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:51.584467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:51.584467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:51.584467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:51.584467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:51.584467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:51.584467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:51.584467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:51.584467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:51.584467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:51.584467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:51.584467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:51.586410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:51.586410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:51.586410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:51.586410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:51.586410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:51.586410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:51.586410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:51.586410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:51.586410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:51.586410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:51.586410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:51.586410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:51.586410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:51.586410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:51.586410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:51.586410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:51.588404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:51.588404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:51.588404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:51.588404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:51.588404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:51.588404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:51.588404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:51.588404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:51.588404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:51.588404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:51.588404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:31:51.588404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:51.588404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 454/898] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Aug 30 14:31:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:51.592035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eeb383-fbd7-4160-930f-c302c6f6d403 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:51.739237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eeb383-fbd7-4160-930f-c302c6f6d403 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:51.742259 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:51.783425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc76b6c-4af4-4242-8973-26c0b33c3d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:51.784540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc76b6c-4af4-4242-8973-26c0b33c3d83 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:51.785030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc76b6c-4af4-4242-8973-26c0b33c3d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:51.785521 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc76b6c-4af4-4242-8973-26c0b33c3d83 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:51.786457 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc76b6c-4af4-4242-8973-26c0b33c3d83 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:51.786972 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc76b6c-4af4-4242-8973-26c0b33c3d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:51.788662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc76b6c-4af4-4242-8973-26c0b33c3d83 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:51.799680 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 445/899] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Aug 30 14:31:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:51.808774 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7e8dd-cb0a-483e-9b4a-0c4bc535184b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:51.847572 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eee932-9ed6-44a6-8896-4834a9b925f8 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:51.848144 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eee932-9ed6-44a6-8896-4834a9b925f8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:51.848144 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eee932-9ed6-44a6-8896-4834a9b925f8 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:51.849774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 446/900] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:51.856069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1413b-5006-4d1e-b846-2c7519baede5 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:51.856721 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1413b-5006-4d1e-b846-2c7519baede5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:51.857193 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1413b-5006-4d1e-b846-2c7519baede5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:51.878155 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-6fb1413b-5006-4d1e-b846-2c7519baede5 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:51.879797 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6fb1413b-5006-4d1e-b846-2c7519baede5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:51.914676 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:51.937003 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7e8dd-cb0a-483e-9b4a-0c4bc535184b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:51.975416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 447/901] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:51 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:31:51.980213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0665921-e9e5-453c-9cd9-1e59fd0307bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:51.981178 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0665921-e9e5-453c-9cd9-1e59fd0307bf None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:51.981647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0665921-e9e5-453c-9cd9-1e59fd0307bf None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:51.982069 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0665921-e9e5-453c-9cd9-1e59fd0307bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:51.996769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0665921-e9e5-453c-9cd9-1e59fd0307bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:52.000323 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0665921-e9e5-453c-9cd9-1e59fd0307bf None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:52.031439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-a0665921-e9e5-453c-9cd9-1e59fd0307bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:52.031439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:52.031439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:52.031439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:52.031439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:52.031439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:52.031439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:52.031439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:52.031439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:52.031439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:52.031439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:52.031439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:52.031439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:52.031439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:52.031439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:52.031439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:52.033257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:52.033257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:52.033257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:52.033257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:52.033257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:52.033257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:52.033257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:52.033257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:52.033257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:52.033257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:52.033257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:52.033257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:52.033257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:52.033257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:52.033257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:52.033257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:52.033257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:52.034645 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:52.034645 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:52.034645 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:52.037912 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 455/902] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:52.047154 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1413b-5006-4d1e-b846-2c7519baede5 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:52.161336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:52.167300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1413b-5006-4d1e-b846-2c7519baede5 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:52.204745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa7c53-2c69-4b49-9261-5003f4ea4486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:52.205382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa7c53-2c69-4b49-9261-5003f4ea4486 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:52.205658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa7c53-2c69-4b49-9261-5003f4ea4486 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:52.205919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa7c53-2c69-4b49-9261-5003f4ea4486 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:52.206944 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa7c53-2c69-4b49-9261-5003f4ea4486 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:52.207240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa7c53-2c69-4b49-9261-5003f4ea4486 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:52.208668 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa7c53-2c69-4b49-9261-5003f4ea4486 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:52.222091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 448/903] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:52.228342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0665921-e9e5-453c-9cd9-1e59fd0307bf None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:52.268358 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa7c53-2c69-4b49-9261-5003f4ea4486 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:52.364732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0665921-e9e5-453c-9cd9-1e59fd0307bf None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:52.402322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa7c53-2c69-4b49-9261-5003f4ea4486 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:52.409345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c09bb-a355-43eb-b5e9-6b4ebaf5d412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:52.410376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c09bb-a355-43eb-b5e9-6b4ebaf5d412 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:52.411444 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c09bb-a355-43eb-b5e9-6b4ebaf5d412 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:52.411956 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c09bb-a355-43eb-b5e9-6b4ebaf5d412 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:52.415285 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:52.425933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c09bb-a355-43eb-b5e9-6b4ebaf5d412 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:52.428635 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c09bb-a355-43eb-b5e9-6b4ebaf5d412 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:52.443200 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d758206-6f61-4154-9024-96c44c506192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:52.443961 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d758206-6f61-4154-9024-96c44c506192 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:52.444404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d758206-6f61-4154-9024-96c44c506192 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:52.444737 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d758206-6f61-4154-9024-96c44c506192 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:52.445001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-851c09bb-a355-43eb-b5e9-6b4ebaf5d412 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:31:52.445001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:52.445001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:52.445001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:52.445001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:52.445001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:52.445001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:52.445001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:52.445001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:52.445001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:52.445001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:52.445001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:52.445001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:52.445001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:52.445001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:52.445001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:52.445925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:52.446882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:52.446882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:52.446882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:52.446882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:52.446882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:52.446882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:52.446882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:52.446882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:52.446882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:31:52.446882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:52.447415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d758206-6f61-4154-9024-96c44c506192 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:52.447415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d758206-6f61-4154-9024-96c44c506192 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:52.454809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 456/904] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Aug 30 14:31:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:52.457783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c09bb-a355-43eb-b5e9-6b4ebaf5d412 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:52.467414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-9d758206-6f61-4154-9024-96c44c506192 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:31:52.467414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:52.467414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:31:52.467414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:31:52.467414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:31:52.467414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:31:52.467414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:31:52.467414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:52.467414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:31:52.467414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:52.467414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:52.467414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:52.467414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:52.467414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:52.467414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:52.467414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:52.467414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:52.468790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:52.468790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:52.468790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:52.468790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:52.468790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:52.468790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:52.468790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:31:52.468790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:31:52.468790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:31:52.468790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:31:52.468790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:52.468790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:52.468790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:52.468790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:52.468790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:52.468790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:52.469637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:52.469637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:52.469637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:52.469637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:52.469637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:52.469637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:52.469637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:52.469637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:52.469637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:52.469637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:52.469637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:31:52.469637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:52.469637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:31:52.469637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:52.472120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 449/905] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:31:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:31:52.477438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d758206-6f61-4154-9024-96c44c506192 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:52.512685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:52.615419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d758206-6f61-4154-9024-96c44c506192 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:52.645367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c09bb-a355-43eb-b5e9-6b4ebaf5d412 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:52.660824 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842a146-369c-4a18-8fa1-76b38d53b905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:52.661892 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842a146-369c-4a18-8fa1-76b38d53b905 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:52.662446 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842a146-369c-4a18-8fa1-76b38d53b905 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:52.662854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842a146-369c-4a18-8fa1-76b38d53b905 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:52.663640 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842a146-369c-4a18-8fa1-76b38d53b905 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:52.664103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842a146-369c-4a18-8fa1-76b38d53b905 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:52.666290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842a146-369c-4a18-8fa1-76b38d53b905 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:52.672949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 450/906] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:31:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:31:52.683234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842a146-369c-4a18-8fa1-76b38d53b905 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:52.709392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5ae53-19c6-44b5-b4ff-dc5263074751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:52.710547 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5ae53-19c6-44b5-b4ff-dc5263074751 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:52.711133 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5ae53-19c6-44b5-b4ff-dc5263074751 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:52.711696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5ae53-19c6-44b5-b4ff-dc5263074751 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:52.713651 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a5ae53-19c6-44b5-b4ff-dc5263074751 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:52.714189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a5ae53-19c6-44b5-b4ff-dc5263074751 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:52.716558 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a5ae53-19c6-44b5-b4ff-dc5263074751 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:52.756022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 457/907] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Aug 30 14:31:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:52.757938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5ae53-19c6-44b5-b4ff-dc5263074751 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:52.814537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:52.835656 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842a146-369c-4a18-8fa1-76b38d53b905 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:52.861133 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:52.880986 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcf05d-332f-4c83-9f93-03b16cee5685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:52.881667 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcf05d-332f-4c83-9f93-03b16cee5685 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:52.882070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcf05d-332f-4c83-9f93-03b16cee5685 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:52.882381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcf05d-332f-4c83-9f93-03b16cee5685 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:52.884113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcf05d-332f-4c83-9f93-03b16cee5685 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:52.884423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcf05d-332f-4c83-9f93-03b16cee5685 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:52.886161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcf05d-332f-4c83-9f93-03b16cee5685 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:52.897828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 451/908] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Aug 30 14:31:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:52.897828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70674b3a-24f0-4b43-bd0f-d9de81eb9402 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:52.897828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70674b3a-24f0-4b43-bd0f-d9de81eb9402 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:52.897828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70674b3a-24f0-4b43-bd0f-d9de81eb9402 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:52.898220 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 452/909] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:52.898974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70674b3a-24f0-4b43-bd0f-d9de81eb9402 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:52.935144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5ae53-19c6-44b5-b4ff-dc5263074751 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:52.991009 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f475a22-ec3f-4553-9324-2f7333a939cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:52.991627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f475a22-ec3f-4553-9324-2f7333a939cb None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:52.991907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f475a22-ec3f-4553-9324-2f7333a939cb None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:52.992184 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f475a22-ec3f-4553-9324-2f7333a939cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:53.052244 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70674b3a-24f0-4b43-bd0f-d9de81eb9402 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:53.077085 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f475a22-ec3f-4553-9324-2f7333a939cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:53.079635 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f475a22-ec3f-4553-9324-2f7333a939cb None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:53.099798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a913d43-401f-4511-946b-79cd59526202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:53.101107 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a913d43-401f-4511-946b-79cd59526202 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:53.101534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a913d43-401f-4511-946b-79cd59526202 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:53.101934 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a913d43-401f-4511-946b-79cd59526202 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:53.102873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a913d43-401f-4511-946b-79cd59526202 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:53.104747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a913d43-401f-4511-946b-79cd59526202 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:53.134741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 453/910] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:31:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:31:53.136794 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412225e-7a58-4f83-b184-646f4aaeb27b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:53.137237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412225e-7a58-4f83-b184-646f4aaeb27b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:53.137648 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412225e-7a58-4f83-b184-646f4aaeb27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:53.151835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:53.155156 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:53.171256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9412225e-7a58-4f83-b184-646f4aaeb27b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:53.173831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9412225e-7a58-4f83-b184-646f4aaeb27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:53.239510 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 458/911] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:52 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:31:53.240639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f475a22-ec3f-4553-9324-2f7333a939cb None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:53.318009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 454/912] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:53 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:53.399511 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45da967-d9c0-4b3a-b066-aea71f1ff729 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:53.399756 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45da967-d9c0-4b3a-b066-aea71f1ff729 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:53.400121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45da967-d9c0-4b3a-b066-aea71f1ff729 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:53.401914 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 455/913] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:31:53.407365 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94f598-8563-4099-80f7-39768bdff815 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:53.407744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94f598-8563-4099-80f7-39768bdff815 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:53.407922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94f598-8563-4099-80f7-39768bdff815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:53.434810 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4b94f598-8563-4099-80f7-39768bdff815 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:53.436355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4b94f598-8563-4099-80f7-39768bdff815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:53.445037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f475a22-ec3f-4553-9324-2f7333a939cb None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:53.507000 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71466d-e963-4751-a477-a900cd36ce38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:53.508380 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71466d-e963-4751-a477-a900cd36ce38 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:53.508844 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71466d-e963-4751-a477-a900cd36ce38 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:53.509678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71466d-e963-4751-a477-a900cd36ce38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:53.531390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71466d-e963-4751-a477-a900cd36ce38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:53.533543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71466d-e963-4751-a477-a900cd36ce38 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:53.540183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 456/914] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:53 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:31:53.552741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-5e71466d-e963-4751-a477-a900cd36ce38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:53.552741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:53.552741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:53.552741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:53.552741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:53.552741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:53.552741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:53.552741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:53.552741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:53.552741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:53.552741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:53.552741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:53.552741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:53.552741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:53.552741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:53.552741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:53.553904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:53.553904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:53.553904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:53.553904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:53.553904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:53.553904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:53.553904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:53.553904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:53.553904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:53.553904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:53.553904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:53.553904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:53.553904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:53.553904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:53.553904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:53.553904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:53.553904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:53.554989 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:53.554989 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:53.554989 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:53.557795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 459/915] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:53.565232 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94f598-8563-4099-80f7-39768bdff815 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:53.641948 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71466d-e963-4751-a477-a900cd36ce38 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:53.646437 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:53.660510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:53.752759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94f598-8563-4099-80f7-39768bdff815 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:53.806693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71466d-e963-4751-a477-a900cd36ce38 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:53.827173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca0c37-a4b8-45dc-b986-a44684baf7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:53.828229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca0c37-a4b8-45dc-b986-a44684baf7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:53.828687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca0c37-a4b8-45dc-b986-a44684baf7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:53.829163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca0c37-a4b8-45dc-b986-a44684baf7c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:53.830304 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca0c37-a4b8-45dc-b986-a44684baf7c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:53.830826 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca0c37-a4b8-45dc-b986-a44684baf7c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:53.833696 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca0c37-a4b8-45dc-b986-a44684baf7c2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:53.854892 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 457/916] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:53.856406 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca0c37-a4b8-45dc-b986-a44684baf7c2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:53.870011 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc77c95-f32d-430b-ac86-92a2c1681d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:53.870986 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc77c95-f32d-430b-ac86-92a2c1681d73 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:53.871466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc77c95-f32d-430b-ac86-92a2c1681d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:53.871911 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc77c95-f32d-430b-ac86-92a2c1681d73 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:53.872941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc77c95-f32d-430b-ac86-92a2c1681d73 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:53.875097 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc77c95-f32d-430b-ac86-92a2c1681d73 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:53.885938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-ffc77c95-f32d-430b-ac86-92a2c1681d73 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:31:53.885938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:53.885938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:31:53.885938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:31:53.885938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:31:53.885938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:31:53.885938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:31:53.885938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:53.885938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:31:53.885938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:53.885938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:53.885938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:53.885938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:53.885938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:53.885938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:53.885938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:53.887824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:53.887824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:53.887824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:53.887824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:53.887824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:53.887824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:53.887824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:53.887824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:31:53.887824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:31:53.887824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:31:53.887824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:31:53.887824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:53.887824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:53.887824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:53.887824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:53.887824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:53.887824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:53.891734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:53.891734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:53.891734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:53.891734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:53.891734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:53.891734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:53.891734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:53.891734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:53.891734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:53.891734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:53.891734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:31:53.891734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:53.891734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:31:53.891734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:53.891734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 460/917] 10.222.0.25 () {56 vars in 1132 bytes} [Tue Aug 30 14:31:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:31:53.894493 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc77c95-f32d-430b-ac86-92a2c1681d73 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:53.981829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:54.026648 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca0c37-a4b8-45dc-b986-a44684baf7c2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:54.031724 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc77c95-f32d-430b-ac86-92a2c1681d73 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:54.052858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:54.073169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1730192-e9e6-465a-99d6-52125e378ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:54.073796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1730192-e9e6-465a-99d6-52125e378ce7 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:54.074060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1730192-e9e6-465a-99d6-52125e378ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:54.074371 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1730192-e9e6-465a-99d6-52125e378ce7 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:54.074971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1730192-e9e6-465a-99d6-52125e378ce7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:54.076332 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6db80a-09cd-43e5-b413-ef7f9f39f015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:54.076626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1730192-e9e6-465a-99d6-52125e378ce7 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:54.077373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6db80a-09cd-43e5-b413-ef7f9f39f015 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:54.077848 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6db80a-09cd-43e5-b413-ef7f9f39f015 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:54.078292 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6db80a-09cd-43e5-b413-ef7f9f39f015 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:54.091154 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6db80a-09cd-43e5-b413-ef7f9f39f015 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:54.096221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 458/918] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Aug 30 14:31:53 2022] POST /identity/v3/services => generated 241 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:31:54.096494 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6db80a-09cd-43e5-b413-ef7f9f39f015 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:54.096646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1730192-e9e6-465a-99d6-52125e378ce7 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:54.120666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-0e6db80a-09cd-43e5-b413-ef7f9f39f015 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:54.120666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:54.120666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:54.120666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:54.120666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:54.120666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:54.120666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:54.120666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:54.120666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:54.120666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:54.120666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:54.120666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:54.120666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:54.120666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:54.120666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:54.122707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:54.122707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:54.122707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:54.122707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:54.122707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:54.122707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:54.122707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:54.122707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:54.122707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:54.122707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:54.122707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:54.122707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:54.122707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:54.122707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:54.122707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:54.122707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:54.122707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:54.125735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:54.125735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:54.125735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:54.125735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:54.125735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:54.125735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:54.125735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:54.125735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:54.136580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 461/919] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:54.140525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6db80a-09cd-43e5-b413-ef7f9f39f015 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:54.287047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1730192-e9e6-465a-99d6-52125e378ce7 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:54.321094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6db80a-09cd-43e5-b413-ef7f9f39f015 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:54.335397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca1dc5-b91e-4037-aa27-ece64871eb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:54.336928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca1dc5-b91e-4037-aa27-ece64871eb6e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:54.336928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca1dc5-b91e-4037-aa27-ece64871eb6e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:54.336928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca1dc5-b91e-4037-aa27-ece64871eb6e None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:54.337408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca1dc5-b91e-4037-aa27-ece64871eb6e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:54.337556 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca1dc5-b91e-4037-aa27-ece64871eb6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:54.338963 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca1dc5-b91e-4037-aa27-ece64871eb6e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:54.345108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 459/920] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Aug 30 14:31:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:54.351883 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca1dc5-b91e-4037-aa27-ece64871eb6e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:54.381016 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:54.396247 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a422bd2a-c397-4e74-89c1-37593fd989df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:54.397252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422bd2a-c397-4e74-89c1-37593fd989df None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:54.397790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422bd2a-c397-4e74-89c1-37593fd989df None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:54.398380 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422bd2a-c397-4e74-89c1-37593fd989df None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:54.399821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a422bd2a-c397-4e74-89c1-37593fd989df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:54.400439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a422bd2a-c397-4e74-89c1-37593fd989df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:54.403149 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a422bd2a-c397-4e74-89c1-37593fd989df None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:54.406913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:54.437494 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 462/921] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:54.444530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a422bd2a-c397-4e74-89c1-37593fd989df None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:54.491836 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca1dc5-b91e-4037-aa27-ece64871eb6e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:54.533080 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b22d0-8259-45f3-84e5-7751bca02a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:54.534026 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b22d0-8259-45f3-84e5-7751bca02a8a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:54.534326 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b22d0-8259-45f3-84e5-7751bca02a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:54.534326 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b22d0-8259-45f3-84e5-7751bca02a8a None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:54.534970 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b22d0-8259-45f3-84e5-7751bca02a8a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:54.535271 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b22d0-8259-45f3-84e5-7751bca02a8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:54.536693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b22d0-8259-45f3-84e5-7751bca02a8a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:54.547595 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 460/922] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Aug 30 14:31:54 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:31:54.550075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b22d0-8259-45f3-84e5-7751bca02a8a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:54.633089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a422bd2a-c397-4e74-89c1-37593fd989df None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:54.678921 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c0ac7-7e97-47b2-99a1-2e5873e519e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:54.679904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c0ac7-7e97-47b2-99a1-2e5873e519e2 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:54.680399 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c0ac7-7e97-47b2-99a1-2e5873e519e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:54.680844 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c0ac7-7e97-47b2-99a1-2e5873e519e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:54.681805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c0ac7-7e97-47b2-99a1-2e5873e519e2 None admin] RBAC: Adding query filter params (role.id=2295f599ff36442faecde157de2c08c8, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=8fc940594a1848aa9644b645049a1bd6) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:54.682314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c0ac7-7e97-47b2-99a1-2e5873e519e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:54.683963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c0ac7-7e97-47b2-99a1-2e5873e519e2 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:54.690743 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 463/923] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:54 2022] GET /identity/v3/role_assignments?user.id=8fc940594a1848aa9644b645049a1bd6&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=2295f599ff36442faecde157de2c08c8 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:54.739905 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b22d0-8259-45f3-84e5-7751bca02a8a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:54.743511 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:54.790481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59496e54-ac26-47ef-bc8c-08e578fe7ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:54.791435 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59496e54-ac26-47ef-bc8c-08e578fe7ff9 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:54.791870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59496e54-ac26-47ef-bc8c-08e578fe7ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:54.792324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59496e54-ac26-47ef-bc8c-08e578fe7ff9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:54.793213 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59496e54-ac26-47ef-bc8c-08e578fe7ff9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:54.795081 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59496e54-ac26-47ef-bc8c-08e578fe7ff9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:54.834707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 461/924] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:31:54 2022] POST /identity/v3/endpoints => generated 347 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:31:55.282643 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6c60c-76b6-41c4-85c6-458d9237ca7e None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:55.283191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6c60c-76b6-41c4-85c6-458d9237ca7e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:55.283191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6c60c-76b6-41c4-85c6-458d9237ca7e None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:55.284813 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 464/925] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:55.290642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f432df2-6050-4f79-8d6c-0d89448ce590 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:55.291157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f432df2-6050-4f79-8d6c-0d89448ce590 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:55.291157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f432df2-6050-4f79-8d6c-0d89448ce590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:55.310848 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-5f432df2-6050-4f79-8d6c-0d89448ce590 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:55.312133 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5f432df2-6050-4f79-8d6c-0d89448ce590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:55.429261 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 462/926] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:55.823768 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6c60c-76b6-41c4-85c6-458d9237ca7e None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:55.903366 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:55.960663 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6c60c-76b6-41c4-85c6-458d9237ca7e None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:56.001976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb85da-593e-4aa2-ac82-3e976a6bbf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:56.002606 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb85da-593e-4aa2-ac82-3e976a6bbf4f None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:56.002882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb85da-593e-4aa2-ac82-3e976a6bbf4f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:56.003152 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb85da-593e-4aa2-ac82-3e976a6bbf4f None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:56.003825 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb85da-593e-4aa2-ac82-3e976a6bbf4f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:56.005314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb85da-593e-4aa2-ac82-3e976a6bbf4f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:56.008064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: WARNING keystone.server.flask.common [None req-dffb85da-593e-4aa2-ac82-3e976a6bbf4f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 14:31:56.029760 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.sql.core [None req-dffb85da-593e-4aa2-ac82-3e976a6bbf4f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 14:31:56.029760 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 14:31:56.029760 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [parameters: {'id': '7870c568c52346bea0c315ea5b15bb1f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 14:31:56.029760 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81733) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 14:31:56.048783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-dffb85da-593e-4aa2-ac82-3e976a6bbf4f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7870c568c52346bea0c315ea5b15bb1f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7870c568c52346bea0c315ea5b15bb1f. Aug 30 14:31:56.048783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:56.048783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:31:56.048783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:31:56.048783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:31:56.048783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:31:56.048783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:31:56.048783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:31:56.048783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:31:56.048783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:31:56.048783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:31:56.048783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:31:56.048783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application result.read() Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 30 14:31:56.049735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 30 14:31:56.050728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:31:56.050728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:31:56.050728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 14:31:56.050728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application yield resource Aug 30 14:31:56.050728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:31:56.050728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:31:56.050728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 14:31:56.050728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 14:31:56.050728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 14:31:56.050728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 14:31:56.050728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 14:31:56.050728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise self.value Aug 30 14:31:56.050728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 14:31:56.050728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 14:31:56.050728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 14:31:56.050728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application session.commit() Aug 30 14:31:56.050728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 14:31:56.052581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 14:31:56.052581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 14:31:56.052581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 14:31:56.052581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 14:31:56.052581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self.session.flush() Aug 30 14:31:56.052581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 14:31:56.052581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 14:31:56.052581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 14:31:56.052581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 14:31:56.052581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 14:31:56.052581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application compat.raise_( Aug 30 14:31:56.052581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:31:56.052581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception Aug 30 14:31:56.052581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 14:31:56.052581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 14:31:56.052581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 14:31:56.052581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 14:31:56.053806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 14:31:56.053806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 14:31:56.053806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 14:31:56.053806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 14:31:56.053806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 14:31:56.053806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 14:31:56.053806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 14:31:56.053806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 14:31:56.053806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 14:31:56.053806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 14:31:56.053806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 14:31:56.053806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 14:31:56.053806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 14:31:56.053806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 14:31:56.053806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 14:31:56.054902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 14:31:56.054902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:31:56.054902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception Aug 30 14:31:56.054902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:31:56.054902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:31:56.054902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:31:56.054902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:31:56.054902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:31:56.054902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:31:56.054902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:31:56.054902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:31:56.054902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:31:56.054902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:31:56.054902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:31:56.054902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application result.read() Aug 30 14:31:56.054902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:31:56.054902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:31:56.057188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:31:56.057188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:31:56.057188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:31:56.057188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:31:56.057188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:31:56.057188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:31:56.057188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 14:31:56.057188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 14:31:56.057188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [parameters: {'id': '7870c568c52346bea0c315ea5b15bb1f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 14:31:56.057188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 14:31:56.057188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:56.057188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:56.057188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:56.057188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:56.057188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 30 14:31:56.059079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 30 14:31:56.059079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 14:31:56.059079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 14:31:56.059079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 30 14:31:56.059079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:56.059079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:56.059079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:56.059079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:56.059079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:56.059079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:56.059079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:56.059079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:56.059079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:56.059079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:56.059079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:56.059079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:56.059079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:56.069888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:56.069888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 30 14:31:56.069888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 30 14:31:56.069888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:56.069888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:56.069888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 30 14:31:56.069888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 30 14:31:56.069888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7870c568c52346bea0c315ea5b15bb1f. Aug 30 14:31:56.069888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:56.069888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 465/927] 10.222.0.25 () {60 vars in 1140 bytes} [Tue Aug 30 14:31:55 2022] POST /identity/v3/projects => generated 262 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:56.071185 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f432df2-6050-4f79-8d6c-0d89448ce590 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:56.148914 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:56.165986 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f432df2-6050-4f79-8d6c-0d89448ce590 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:56.207988 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628caf88-c52a-4db2-b654-11887f58cdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:56.208686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628caf88-c52a-4db2-b654-11887f58cdde None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:56.208960 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628caf88-c52a-4db2-b654-11887f58cdde None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:56.209230 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628caf88-c52a-4db2-b654-11887f58cdde None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:56.217382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628caf88-c52a-4db2-b654-11887f58cdde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:56.219225 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628caf88-c52a-4db2-b654-11887f58cdde None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:56.231685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-628caf88-c52a-4db2-b654-11887f58cdde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:56.231685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:56.231685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:56.231685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:56.231685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:56.231685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:56.231685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:56.231685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:56.231685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:56.231685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:56.231685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:56.231685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:56.231685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:56.231685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:56.231685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:56.233853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:56.233853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:56.233853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:56.233853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:56.233853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:56.233853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:56.233853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:56.233853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:56.233853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:56.233853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:56.233853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:56.233853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:56.233853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:56.233853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:56.233853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:56.233853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:56.233853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:56.235894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:56.235894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:56.235894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:56.235894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:56.235894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:56.235894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:56.235894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:56.235894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:56.235894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 463/928] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:31:56.239678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb85da-593e-4aa2-ac82-3e976a6bbf4f None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:56.353896 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb85da-593e-4aa2-ac82-3e976a6bbf4f None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:56.364619 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:56.392576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c9b88-06f3-4e19-9e13-8d0d60edc608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:56.393884 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c9b88-06f3-4e19-9e13-8d0d60edc608 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:56.394166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c9b88-06f3-4e19-9e13-8d0d60edc608 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:56.394419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c9b88-06f3-4e19-9e13-8d0d60edc608 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:56.395158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c9b88-06f3-4e19-9e13-8d0d60edc608 None admin] RBAC: Adding query filter params (name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:56.395454 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c9b88-06f3-4e19-9e13-8d0d60edc608 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:56.397174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c9b88-06f3-4e19-9e13-8d0d60edc608 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:56.411377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58eb1e3-dd55-4741-8cb9-8de97f962014 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:56.412078 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58eb1e3-dd55-4741-8cb9-8de97f962014 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:56.412399 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58eb1e3-dd55-4741-8cb9-8de97f962014 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:56.419859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 466/930] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Aug 30 14:31:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:56.420240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 464/930] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:56.423732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fa895-ab0a-43d6-9552-827c36087b91 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:56.424042 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fa895-ab0a-43d6-9552-827c36087b91 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:56.424317 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fa895-ab0a-43d6-9552-827c36087b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:56.446879 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-7a4fa895-ab0a-43d6-9552-827c36087b91 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:56.448575 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7a4fa895-ab0a-43d6-9552-827c36087b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:56.532128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 467/931] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:31:56.808611 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58eb1e3-dd55-4741-8cb9-8de97f962014 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:56.929630 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58eb1e3-dd55-4741-8cb9-8de97f962014 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:56.939060 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7c751-f32f-4f59-b8b2-88d1382668b3 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:56.939717 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7c751-f32f-4f59-b8b2-88d1382668b3 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:56.940244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7c751-f32f-4f59-b8b2-88d1382668b3 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:56.942124 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 468/932] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:56.948382 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa48074-e8a7-4a2d-b81f-64c17285d69a None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:56.949605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa48074-e8a7-4a2d-b81f-64c17285d69a None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:56.950137 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa48074-e8a7-4a2d-b81f-64c17285d69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:56.973400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e773ce5-7252-4e47-84e6-d4eb1bb4f691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:56.974064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e773ce5-7252-4e47-84e6-d4eb1bb4f691 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:56.974330 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e773ce5-7252-4e47-84e6-d4eb1bb4f691 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:56.974595 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e773ce5-7252-4e47-84e6-d4eb1bb4f691 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:56.975184 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e773ce5-7252-4e47-84e6-d4eb1bb4f691 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:56.976602 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e773ce5-7252-4e47-84e6-d4eb1bb4f691 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:56.979277 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-7fa48074-e8a7-4a2d-b81f-64c17285d69a None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:56.981552 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7fa48074-e8a7-4a2d-b81f-64c17285d69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:56.987596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-3e773ce5-7252-4e47-84e6-d4eb1bb4f691 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:31:56.987596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:56.987596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:31:56.987596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:31:56.987596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:31:56.987596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:31:56.987596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:31:56.987596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:56.987596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:31:56.987596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:56.987596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:56.987596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:56.987596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:56.987596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:56.987596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:56.987596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:56.987596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:56.989049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:56.989049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:56.989049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:56.989049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:56.989049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:56.989049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:56.989049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:31:56.989049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:31:56.989049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:31:56.989049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:31:56.989049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:56.989049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:56.989049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:56.989049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:56.989049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:56.989049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:56.989928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:56.989928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:56.989928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:56.989928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:56.989928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:56.989928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:56.989928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:56.989928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:56.989928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:56.989928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:56.989928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:31:56.989928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:56.989928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:31:56.989928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:56.989928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 465/933] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Aug 30 14:31:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:56.995743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e773ce5-7252-4e47-84e6-d4eb1bb4f691 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:57.010964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:57.088082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 469/934] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:31:57.088082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426de09e-0abd-42ba-bd3d-78c7c7ac0e51 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:57.088082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426de09e-0abd-42ba-bd3d-78c7c7ac0e51 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:57.088082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426de09e-0abd-42ba-bd3d-78c7c7ac0e51 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:57.088082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 470/935] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:57.095404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40750e62-9fea-46d0-b6a4-9736811945bf None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:57.095687 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40750e62-9fea-46d0-b6a4-9736811945bf None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:57.095946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40750e62-9fea-46d0-b6a4-9736811945bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:57.121526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-40750e62-9fea-46d0-b6a4-9736811945bf None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:57.123315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-40750e62-9fea-46d0-b6a4-9736811945bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:57.136040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e773ce5-7252-4e47-84e6-d4eb1bb4f691 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:57.172875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:57.178894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b272df-66f8-45e7-9073-7f1dc8141601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:57.179512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b272df-66f8-45e7-9073-7f1dc8141601 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:57.179788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b272df-66f8-45e7-9073-7f1dc8141601 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:57.180083 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b272df-66f8-45e7-9073-7f1dc8141601 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:57.180724 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b272df-66f8-45e7-9073-7f1dc8141601 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:57.181021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b272df-66f8-45e7-9073-7f1dc8141601 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:57.182410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b272df-66f8-45e7-9073-7f1dc8141601 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:57.195779 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 466/936] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Aug 30 14:31:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:31:57.198751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b272df-66f8-45e7-9073-7f1dc8141601 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:57.248186 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 471/937] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:31:57.344149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:57.351889 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40750e62-9fea-46d0-b6a4-9736811945bf None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:57.379686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b272df-66f8-45e7-9073-7f1dc8141601 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:57.452381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cee807-8308-4607-a2b8-43d5fca2f282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:57.454889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cee807-8308-4607-a2b8-43d5fca2f282 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:57.455611 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cee807-8308-4607-a2b8-43d5fca2f282 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:57.456939 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cee807-8308-4607-a2b8-43d5fca2f282 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:57.459070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cee807-8308-4607-a2b8-43d5fca2f282 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:57.460118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cee807-8308-4607-a2b8-43d5fca2f282 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:57.462711 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cee807-8308-4607-a2b8-43d5fca2f282 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:57.473975 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 467/938] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Aug 30 14:31:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:31:57.482726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cee807-8308-4607-a2b8-43d5fca2f282 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:57.539804 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40750e62-9fea-46d0-b6a4-9736811945bf None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:57.609667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd67ef2-07f0-4c8d-b8d1-a3f993257afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:57.610218 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd67ef2-07f0-4c8d-b8d1-a3f993257afc None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:57.610497 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd67ef2-07f0-4c8d-b8d1-a3f993257afc None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:57.610938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd67ef2-07f0-4c8d-b8d1-a3f993257afc None admin] PATH_INFO: `/v3/services/compute` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:57.611789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd67ef2-07f0-4c8d-b8d1-a3f993257afc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:57.614162 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd67ef2-07f0-4c8d-b8d1-a3f993257afc None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:57.629568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-6cd67ef2-07f0-4c8d-b8d1-a3f993257afc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:31:57.629568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:57.629568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:31:57.629568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:31:57.629568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:31:57.629568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:31:57.629568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:31:57.629568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:57.629568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:31:57.629568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:57.629568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:57.629568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:57.629568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:57.629568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:57.629568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:57.629568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:57.629568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:57.630561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:57.630561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:57.630561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:57.630561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:57.630561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:57.630561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:57.630561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:31:57.630561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:31:57.630561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:31:57.630561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:31:57.630561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:57.630561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:57.630561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:57.630561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:57.630561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:57.630561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:57.631392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:57.631392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:57.631392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:57.631392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:57.631392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:57.631392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:57.631392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:57.631392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:57.631392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:57.631392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:57.631392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:31:57.631392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:31:57.631392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:31:57.631392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:57.631392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 472/939] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:31:57.632965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd67ef2-07f0-4c8d-b8d1-a3f993257afc None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:57.655872 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cee807-8308-4607-a2b8-43d5fca2f282 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:57.678471 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:57.701578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7ee03-3c82-4cb8-9b8e-1ecba3582aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:57.702525 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7ee03-3c82-4cb8-9b8e-1ecba3582aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:57.702967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7ee03-3c82-4cb8-9b8e-1ecba3582aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:57.703393 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7ee03-3c82-4cb8-9b8e-1ecba3582aa6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:57.704601 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7ee03-3c82-4cb8-9b8e-1ecba3582aa6 None admin] RBAC: Adding query filter params (interface=public, service_id=dd0695a6299a44d8a4c1433fedfd1e28, region_id=RegionOne) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:57.704601 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7ee03-3c82-4cb8-9b8e-1ecba3582aa6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:57.708478 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7ee03-3c82-4cb8-9b8e-1ecba3582aa6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:57.716375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 468/940] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Aug 30 14:31:57 2022] GET /identity/v3/endpoints?service_id=dd0695a6299a44d8a4c1433fedfd1e28&interface=public®ion_id=RegionOne => generated 194 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:31:57.717369 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7ee03-3c82-4cb8-9b8e-1ecba3582aa6 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:57.723812 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:57.795256 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd67ef2-07f0-4c8d-b8d1-a3f993257afc None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:57.841590 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65aa240-19f4-466d-9362-2504b9ac7622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:57.842268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aa240-19f4-466d-9362-2504b9ac7622 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:57.842547 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aa240-19f4-466d-9362-2504b9ac7622 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:57.842820 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aa240-19f4-466d-9362-2504b9ac7622 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:57.849544 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7ee03-3c82-4cb8-9b8e-1ecba3582aa6 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:57.854391 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65aa240-19f4-466d-9362-2504b9ac7622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:57.859939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-c65aa240-19f4-466d-9362-2504b9ac7622 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:31:57.859939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:57.859939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:57.859939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:57.859939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:57.859939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:57.859939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:57.859939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:57.859939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:57.859939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:57.859939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:57.859939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:57.859939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:57.859939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:57.859939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:57.859939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:57.861789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:57.861789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:57.861789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:57.861789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:57.861789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:57.861789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:57.861789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:57.861789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:57.861789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:57.861789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:57.861789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:57.861789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:57.861789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:57.861789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:57.861789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:57.861789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:57.863902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:57.863902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:57.863902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:57.863902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:57.863902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:57.863902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:57.863902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:57.863902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:57.863902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:57.863902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:57.863902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:31:57.863902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:57.863902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 473/941] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Aug 30 14:31:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:57.863902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65aa240-19f4-466d-9362-2504b9ac7622 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:57.904428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd9ad9-427d-452b-8424-03ab8e13272d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:57.905181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd9ad9-427d-452b-8424-03ab8e13272d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:57.905532 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd9ad9-427d-452b-8424-03ab8e13272d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:57.905771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd9ad9-427d-452b-8424-03ab8e13272d None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:57.906435 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd9ad9-427d-452b-8424-03ab8e13272d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:57.906734 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd9ad9-427d-452b-8424-03ab8e13272d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:57.908153 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd9ad9-427d-452b-8424-03ab8e13272d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:57.913480 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 469/942] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Aug 30 14:31:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:57.914063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd9ad9-427d-452b-8424-03ab8e13272d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:57.975379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:57.995758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:57.999559 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65aa240-19f4-466d-9362-2504b9ac7622 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:58.039822 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52becd4b-21c4-41b6-a182-23e6d333425c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:58.040457 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52becd4b-21c4-41b6-a182-23e6d333425c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:58.040729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52becd4b-21c4-41b6-a182-23e6d333425c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:58.040830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd9ad9-427d-452b-8424-03ab8e13272d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:58.041004 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52becd4b-21c4-41b6-a182-23e6d333425c None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:58.041601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52becd4b-21c4-41b6-a182-23e6d333425c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:58.042965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52becd4b-21c4-41b6-a182-23e6d333425c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:58.049743 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 474/943] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Aug 30 14:31:57 2022] GET /identity/v3/services => generated 1497 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:31:58.057849 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52becd4b-21c4-41b6-a182-23e6d333425c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:58.109927 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bb7ba-e06d-49c0-a8e9-e600c93b6e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:58.111243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bb7ba-e06d-49c0-a8e9-e600c93b6e91 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:58.111709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bb7ba-e06d-49c0-a8e9-e600c93b6e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:58.112221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bb7ba-e06d-49c0-a8e9-e600c93b6e91 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:58.113121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557bb7ba-e06d-49c0-a8e9-e600c93b6e91 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:58.113610 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557bb7ba-e06d-49c0-a8e9-e600c93b6e91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:58.115583 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557bb7ba-e06d-49c0-a8e9-e600c93b6e91 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:58.128122 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 470/944] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Aug 30 14:31:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:58.141141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bb7ba-e06d-49c0-a8e9-e600c93b6e91 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:58.253723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52becd4b-21c4-41b6-a182-23e6d333425c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:58.271659 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bb7ba-e06d-49c0-a8e9-e600c93b6e91 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:58.301033 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868286e5-9b95-4031-95ad-792bf27d81d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:58.302034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868286e5-9b95-4031-95ad-792bf27d81d4 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:58.302779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868286e5-9b95-4031-95ad-792bf27d81d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:58.303678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868286e5-9b95-4031-95ad-792bf27d81d4 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:58.304810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868286e5-9b95-4031-95ad-792bf27d81d4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:58.305288 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868286e5-9b95-4031-95ad-792bf27d81d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:58.307284 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:58.308695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868286e5-9b95-4031-95ad-792bf27d81d4 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:58.321091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcdcb7-5c54-4aeb-8b38-ebb66536aacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:58.321091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcdcb7-5c54-4aeb-8b38-ebb66536aacf None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:58.321091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcdcb7-5c54-4aeb-8b38-ebb66536aacf None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:58.321091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcdcb7-5c54-4aeb-8b38-ebb66536aacf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:58.322314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 475/945] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Aug 30 14:31:58 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:58.337026 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcdcb7-5c54-4aeb-8b38-ebb66536aacf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:58.338830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcdcb7-5c54-4aeb-8b38-ebb66536aacf None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:58.369863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-99fcdcb7-5c54-4aeb-8b38-ebb66536aacf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:58.369863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:58.369863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:58.369863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:58.369863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:58.369863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:58.369863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:58.369863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:58.369863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:58.369863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:58.369863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:58.369863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:58.369863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:58.369863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:58.369863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:58.369863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:58.370919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:58.370919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:58.370919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:58.370919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:58.370919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:58.370919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:58.370919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:58.370919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:58.370919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:58.370919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:58.370919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:58.370919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:58.370919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:58.370919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:58.370919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:58.370919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:58.370919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:58.371819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:58.371819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:58.371819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:58.372018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 471/946] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:58.380014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868286e5-9b95-4031-95ad-792bf27d81d4 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:58.396674 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:58.505498 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868286e5-9b95-4031-95ad-792bf27d81d4 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:58.544200 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394ff7d-3042-4b24-a37c-64b53dc471a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:58.545704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394ff7d-3042-4b24-a37c-64b53dc471a1 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:58.546033 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394ff7d-3042-4b24-a37c-64b53dc471a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:58.546326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394ff7d-3042-4b24-a37c-64b53dc471a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:58.547017 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394ff7d-3042-4b24-a37c-64b53dc471a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:58.547291 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394ff7d-3042-4b24-a37c-64b53dc471a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:58.548702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394ff7d-3042-4b24-a37c-64b53dc471a1 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:58.565214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 476/947] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:58.573201 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcdcb7-5c54-4aeb-8b38-ebb66536aacf None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:58.657237 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3996eb-215f-4395-b503-d895b0210cd0 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:58.657501 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3996eb-215f-4395-b503-d895b0210cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:58.657780 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3996eb-215f-4395-b503-d895b0210cd0 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:58.659429 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 477/948] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:31:58.664874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e223fb-ff01-43be-b584-1f7eb9fe726c None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:58.665159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e223fb-ff01-43be-b584-1f7eb9fe726c None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:58.665415 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e223fb-ff01-43be-b584-1f7eb9fe726c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:58.688915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-c4e223fb-ff01-43be-b584-1f7eb9fe726c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:58.688915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c4e223fb-ff01-43be-b584-1f7eb9fe726c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:58.718836 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcdcb7-5c54-4aeb-8b38-ebb66536aacf None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:58.769212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d87f5-cf5c-44a4-b8b2-6c14e845329e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:58.771050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d87f5-cf5c-44a4-b8b2-6c14e845329e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:58.771539 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d87f5-cf5c-44a4-b8b2-6c14e845329e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:58.772161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d87f5-cf5c-44a4-b8b2-6c14e845329e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:58.784541 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d87f5-cf5c-44a4-b8b2-6c14e845329e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:58.787362 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d87f5-cf5c-44a4-b8b2-6c14e845329e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:58.790604 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:58.803681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-226d87f5-cf5c-44a4-b8b2-6c14e845329e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:31:58.803681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:58.803681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:58.803681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:58.803681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:58.803681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:58.803681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:58.803681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:58.803681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:58.803681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:58.803681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:58.803681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:58.803681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:58.803681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:58.803681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:58.803681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:58.804971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:58.805947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:58.805947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:58.805947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:58.805947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:58.805947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:58.805947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:58.805947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:58.805947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:58.805947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:31:58.805947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:58.806530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 478/949] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:31:58.814194 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 472/950] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Aug 30 14:31:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:58.816214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e223fb-ff01-43be-b584-1f7eb9fe726c None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:58.947737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e223fb-ff01-43be-b584-1f7eb9fe726c None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:58.990455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3a1be-ff6e-4c3b-b731-d8a115214d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:58.991396 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3a1be-ff6e-4c3b-b731-d8a115214d98 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:58.991854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3a1be-ff6e-4c3b-b731-d8a115214d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:58.992325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3a1be-ff6e-4c3b-b731-d8a115214d98 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:58.993344 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3a1be-ff6e-4c3b-b731-d8a115214d98 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:58.993877 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3a1be-ff6e-4c3b-b731-d8a115214d98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:58.995518 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3a1be-ff6e-4c3b-b731-d8a115214d98 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:59.022702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 479/951] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Aug 30 14:31:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:59.024432 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d87f5-cf5c-44a4-b8b2-6c14e845329e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:59.074806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:59.086478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3a1be-ff6e-4c3b-b731-d8a115214d98 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:59.163069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d87f5-cf5c-44a4-b8b2-6c14e845329e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:59.191917 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:59.209340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0678d-1101-4df4-a515-fc34910bc13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:59.209933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0678d-1101-4df4-a515-fc34910bc13d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:59.210210 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0678d-1101-4df4-a515-fc34910bc13d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:59.210486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0678d-1101-4df4-a515-fc34910bc13d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:59.229118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0678d-1101-4df4-a515-fc34910bc13d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:59.231989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0678d-1101-4df4-a515-fc34910bc13d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:59.250638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3a1be-ff6e-4c3b-b731-d8a115214d98 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:59.257461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-3ae0678d-1101-4df4-a515-fc34910bc13d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:59.257461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:59.257461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:59.257461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:59.257461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:59.257461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:59.257461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:59.257461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:59.257461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:59.257461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:59.257461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:59.257461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:59.257461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:59.257461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:59.257461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:59.257461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:59.258676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:59.258676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:59.258676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:59.258676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:59.258676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:59.258676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:59.258676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:59.258676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:59.258676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:59.258676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:59.258676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:59.258676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:59.258676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:59.258676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:59.258676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:59.258676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:59.258676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:31:59.259767 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:59.259767 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:31:59.259767 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:59.264503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 473/952] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:59.267406 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0678d-1101-4df4-a515-fc34910bc13d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:59.291585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ac5fb-0650-47d2-9718-cdf032f7995d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:59.292242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ac5fb-0650-47d2-9718-cdf032f7995d None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:59.292520 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ac5fb-0650-47d2-9718-cdf032f7995d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:59.292789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ac5fb-0650-47d2-9718-cdf032f7995d None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:59.293443 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ac5fb-0650-47d2-9718-cdf032f7995d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:59.294942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ac5fb-0650-47d2-9718-cdf032f7995d None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:59.311546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.sql.core [None req-d70ac5fb-0650-47d2-9718-cdf032f7995d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 14:31:59.311546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 14:31:59.311546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [parameters: {'user_id': '7fdf09f3b3534ae285db1573d7d3b82c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 14:31:59.311546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81733) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 14:31:59.320113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-d70ac5fb-0650-47d2-9718-cdf032f7995d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 14:31:59.320113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:59.320113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:31:59.320113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:31:59.320113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:31:59.320113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:31:59.320113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:31:59.320113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:31:59.320113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:31:59.320113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:31:59.320113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:31:59.320113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:31:59.320113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:31:59.320113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application result.read() Aug 30 14:31:59.320113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:31:59.322963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:31:59.325252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 14:31:59.325252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application yield resource Aug 30 14:31:59.325252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:31:59.325252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:31:59.325252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 14:31:59.325252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 14:31:59.325252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 14:31:59.325252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 14:31:59.325252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 14:31:59.325252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise self.value Aug 30 14:31:59.325252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 14:31:59.325252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 14:31:59.325252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 14:31:59.325252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application session.commit() Aug 30 14:31:59.325252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 14:31:59.325252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 14:31:59.325252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 14:31:59.335733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 14:31:59.335733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 14:31:59.335733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self.session.flush() Aug 30 14:31:59.335733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 14:31:59.335733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 14:31:59.335733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 14:31:59.335733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 14:31:59.335733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 14:31:59.335733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application compat.raise_( Aug 30 14:31:59.335733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:31:59.335733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception Aug 30 14:31:59.335733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 14:31:59.335733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 14:31:59.335733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 14:31:59.335733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 14:31:59.335733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 14:31:59.335733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 14:31:59.339180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 14:31:59.339180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 14:31:59.339180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 30 14:31:59.339180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 30 14:31:59.339180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 14:31:59.339180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 14:31:59.339180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 14:31:59.339180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 14:31:59.339180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 14:31:59.339180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 14:31:59.339180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 14:31:59.339180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 14:31:59.339180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 14:31:59.339180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 14:31:59.339180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:31:59.339180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception Aug 30 14:31:59.342446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:31:59.342446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:31:59.342446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:31:59.342446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:31:59.342446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:31:59.342446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:31:59.342446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:31:59.342446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:31:59.342446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:31:59.342446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:31:59.342446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:31:59.342446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application result.read() Aug 30 14:31:59.342446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:31:59.342446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:31:59.342446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:31:59.342446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:31:59.350626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:31:59.350626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:31:59.350626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:31:59.350626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:31:59.350626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 14:31:59.350626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 14:31:59.350626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [parameters: {'user_id': '7fdf09f3b3534ae285db1573d7d3b82c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 14:31:59.350626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 14:31:59.350626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:59.350626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:59.350626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:59.350626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:59.350626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:59.350626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:59.350626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:59.357043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:59.357043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:59.357043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:59.357043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:59.357043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:59.357043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:59.357043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:59.357043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 30 14:31:59.357043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 30 14:31:59.357043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:59.357043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:59.357043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:59.357043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:59.357043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:59.357043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:59.357043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 30 14:31:59.357043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 30 14:31:59.360594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 30 14:31:59.360594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 30 14:31:59.360594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 14:31:59.360594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 14:31:59.360594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 14:31:59.360594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:59.360594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 480/953] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Aug 30 14:31:59 2022] POST /identity/v3/users => generated 160 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:31:59.360594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ac5fb-0650-47d2-9718-cdf032f7995d None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:59.382650 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:59.393012 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0678d-1101-4df4-a515-fc34910bc13d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:59.430347 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23ce1a-edd4-4e68-88e9-6b18cd3c6ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:59.430631 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23ce1a-edd4-4e68-88e9-6b18cd3c6ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:59.430933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23ce1a-edd4-4e68-88e9-6b18cd3c6ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:59.431169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23ce1a-edd4-4e68-88e9-6b18cd3c6ff3 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:59.431816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23ce1a-edd4-4e68-88e9-6b18cd3c6ff3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:59.432141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23ce1a-edd4-4e68-88e9-6b18cd3c6ff3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:59.433605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23ce1a-edd4-4e68-88e9-6b18cd3c6ff3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:59.451679 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 474/954] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:59.453319 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23ce1a-edd4-4e68-88e9-6b18cd3c6ff3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:59.537087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ac5fb-0650-47d2-9718-cdf032f7995d None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:59.564239 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:59.569068 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23ce1a-edd4-4e68-88e9-6b18cd3c6ff3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:59.590301 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7992a-11fc-42da-8a11-7c6685599ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:59.591755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7992a-11fc-42da-8a11-7c6685599ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:59.592604 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7992a-11fc-42da-8a11-7c6685599ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:59.593325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7992a-11fc-42da-8a11-7c6685599ed3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:59.604316 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7992a-11fc-42da-8a11-7c6685599ed3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:59.608412 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7992a-11fc-42da-8a11-7c6685599ed3 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:59.611458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dac01f-1b98-4017-8bbe-4e51b287fe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:59.612807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dac01f-1b98-4017-8bbe-4e51b287fe6d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:59.614061 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dac01f-1b98-4017-8bbe-4e51b287fe6d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:59.614061 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dac01f-1b98-4017-8bbe-4e51b287fe6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:59.621984 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dac01f-1b98-4017-8bbe-4e51b287fe6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:59.624522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dac01f-1b98-4017-8bbe-4e51b287fe6d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:59.627750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-ced7992a-11fc-42da-8a11-7c6685599ed3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:31:59.627750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:59.627750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:59.627750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:59.627750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:59.627750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:59.627750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:59.627750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:59.627750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:59.627750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:59.627750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:59.627750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:59.627750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:31:59.627750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:31:59.627750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:31:59.627750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:59.628744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:59.629728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:59.629728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:59.629728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:31:59.629728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:31:59.629728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:31:59.629728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:31:59.629728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:31:59.629728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:31:59.629728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:31:59.629728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:31:59.637548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 481/955] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:31:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:59.638966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7992a-11fc-42da-8a11-7c6685599ed3 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:59.646359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-a0dac01f-1b98-4017-8bbe-4e51b287fe6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:59.646359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:59.646359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:59.646359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:59.646359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:59.646359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:59.646359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:59.646359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:59.646359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:59.646359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:59.646359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:59.646359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:59.646359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:31:59.646359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:31:59.646359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:31:59.647398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:31:59.647398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:59.647398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:59.647398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:59.647398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:59.647398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:59.647398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:59.647398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:59.647398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:59.647398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:59.647398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:59.647398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:59.647398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:59.647398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:59.647398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:59.647398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:59.647398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:59.648390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:31:59.648390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:31:59.648390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:59.648390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:59.648390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:59.648390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:59.648390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:31:59.648390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:31:59.653545 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 475/956] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:31:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:59.655950 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dac01f-1b98-4017-8bbe-4e51b287fe6d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:59.682500 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:59.771123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7992a-11fc-42da-8a11-7c6685599ed3 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:59.811623 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04150c84-b06a-40db-a94e-f6717bb40166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:59.812748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04150c84-b06a-40db-a94e-f6717bb40166 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:59.813204 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04150c84-b06a-40db-a94e-f6717bb40166 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:59.813741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04150c84-b06a-40db-a94e-f6717bb40166 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:59.814640 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04150c84-b06a-40db-a94e-f6717bb40166 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:59.815162 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04150c84-b06a-40db-a94e-f6717bb40166 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:59.816934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04150c84-b06a-40db-a94e-f6717bb40166 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:59.829957 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dac01f-1b98-4017-8bbe-4e51b287fe6d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:59.867630 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc303c13-70d2-43af-93b6-daef336bd9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:59.868464 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc303c13-70d2-43af-93b6-daef336bd9db None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:59.868666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc303c13-70d2-43af-93b6-daef336bd9db None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:59.868792 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 482/957] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Aug 30 14:31:59 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:59.868930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc303c13-70d2-43af-93b6-daef336bd9db None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:59.869653 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc303c13-70d2-43af-93b6-daef336bd9db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:59.869955 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc303c13-70d2-43af-93b6-daef336bd9db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:59.871475 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc303c13-70d2-43af-93b6-daef336bd9db None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:59.892209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 476/958] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:31:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:59.894090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04150c84-b06a-40db-a94e-f6717bb40166 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:59.947737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:00.032669 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04150c84-b06a-40db-a94e-f6717bb40166 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:00.075214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93e430-b950-4c82-86bf-384f6eafd16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:00.076380 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93e430-b950-4c82-86bf-384f6eafd16c None admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:00.076903 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93e430-b950-4c82-86bf-384f6eafd16c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:00.077372 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93e430-b950-4c82-86bf-384f6eafd16c None admin] PATH_INFO: `/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/8fc940594a1848aa9644b645049a1bd6/roles/2295f599ff36442faecde157de2c08c8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:00.080863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93e430-b950-4c82-86bf-384f6eafd16c None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9c57a581324217ad2e64a0c9c7fdee, user_id=8fc940594a1848aa9644b645049a1bd6, role_id=2295f599ff36442faecde157de2c08c8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:00.083044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93e430-b950-4c82-86bf-384f6eafd16c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:00.103079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 483/959] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:31:59 2022] PUT /identity/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/8fc940594a1848aa9644b645049a1bd6/roles/2295f599ff36442faecde157de2c08c8 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:32:00.208165 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65218445-9585-4988-b6d7-bf5ec03ada0e None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:00.208650 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65218445-9585-4988-b6d7-bf5ec03ada0e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:00.209134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65218445-9585-4988-b6d7-bf5ec03ada0e None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:00.211874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 477/960] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:00.221749 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160730c2-6c0a-4436-8028-71880d077925 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:00.222257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160730c2-6c0a-4436-8028-71880d077925 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:00.222755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160730c2-6c0a-4436-8028-71880d077925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:00.253338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-160730c2-6c0a-4436-8028-71880d077925 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:00.255051 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-160730c2-6c0a-4436-8028-71880d077925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:00.339707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 484/961] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:00.546606 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e87ce9-a9ab-495c-9397-24d5cc52ca54 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:00.546824 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e87ce9-a9ab-495c-9397-24d5cc52ca54 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:00.547212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e87ce9-a9ab-495c-9397-24d5cc52ca54 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:00.548588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 478/962] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:32:00.554448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee1866-106c-4013-936f-3a62b310ce2c None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:00.554808 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee1866-106c-4013-936f-3a62b310ce2c None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:00.555089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee1866-106c-4013-936f-3a62b310ce2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:00.574587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-8eee1866-106c-4013-936f-3a62b310ce2c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:00.575807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-8eee1866-106c-4013-936f-3a62b310ce2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:00.606706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e87ce9-a9ab-495c-9397-24d5cc52ca54 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:00.611003 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:00.668091 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 485/963] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:00.687280 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:00.724445 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e87ce9-a9ab-495c-9397-24d5cc52ca54 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:00.760690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcfbcb-f985-465c-b625-9c8176eca1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:00.761429 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcfbcb-f985-465c-b625-9c8176eca1d0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:00.761769 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcfbcb-f985-465c-b625-9c8176eca1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:00.762105 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcfbcb-f985-465c-b625-9c8176eca1d0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:00.762762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dcfbcb-f985-465c-b625-9c8176eca1d0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:00.765091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dcfbcb-f985-465c-b625-9c8176eca1d0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:00.774552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-45dcfbcb-f985-465c-b625-9c8176eca1d0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:32:00.774552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:00.774552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:32:00.774552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:32:00.774552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:32:00.774552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:32:00.774552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:32:00.774552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:32:00.774552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:32:00.774552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:00.774552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:32:00.774552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:00.774552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:00.774552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:00.774552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:00.774552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:00.774552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:00.775911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:00.775911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:00.775911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:00.775911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:00.775911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:00.775911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:00.775911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:32:00.775911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:32:00.775911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:32:00.775911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:32:00.775911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:00.775911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:00.775911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:00.775911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:00.775911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:00.775911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:00.777006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:00.777006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:00.777006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:00.777006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:00.777006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:00.777006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:00.777006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:00.777006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:00.777006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:00.777006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:00.777006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:32:00.777006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:32:00.777006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:32:00.777006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:00.778865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 479/964] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Aug 30 14:32:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:32:00.785496 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee1866-106c-4013-936f-3a62b310ce2c None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:00.898843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee1866-106c-4013-936f-3a62b310ce2c None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:00.935775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcfbcb-f985-465c-b625-9c8176eca1d0 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:00.938111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc892eb-22ea-493a-9dc7-b38595e6e39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:00.938805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc892eb-22ea-493a-9dc7-b38595e6e39b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:00.939148 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc892eb-22ea-493a-9dc7-b38595e6e39b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:00.939483 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc892eb-22ea-493a-9dc7-b38595e6e39b None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:00.940159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc892eb-22ea-493a-9dc7-b38595e6e39b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:00.940991 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc892eb-22ea-493a-9dc7-b38595e6e39b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:00.942377 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc892eb-22ea-493a-9dc7-b38595e6e39b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:00.952431 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 486/965] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Aug 30 14:32:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:00.953356 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc892eb-22ea-493a-9dc7-b38595e6e39b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:00.981819 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:01.053250 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcfbcb-f985-465c-b625-9c8176eca1d0 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:01.075412 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc892eb-22ea-493a-9dc7-b38595e6e39b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:01.088388 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daae1f1-4609-42c6-8a1d-076b63646d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:01.089009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daae1f1-4609-42c6-8a1d-076b63646d08 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:01.089288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daae1f1-4609-42c6-8a1d-076b63646d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:01.089551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daae1f1-4609-42c6-8a1d-076b63646d08 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:01.090210 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daae1f1-4609-42c6-8a1d-076b63646d08 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:01.091655 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daae1f1-4609-42c6-8a1d-076b63646d08 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:01.096196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:01.110188 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 480/966] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Aug 30 14:32:00 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:32:01.132791 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d93f4-3ff3-43b9-aab0-721e9d236086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:01.133466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d93f4-3ff3-43b9-aab0-721e9d236086 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:01.133739 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d93f4-3ff3-43b9-aab0-721e9d236086 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:01.134007 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d93f4-3ff3-43b9-aab0-721e9d236086 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:01.134677 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d93f4-3ff3-43b9-aab0-721e9d236086 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:01.134963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d93f4-3ff3-43b9-aab0-721e9d236086 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:01.136432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d93f4-3ff3-43b9-aab0-721e9d236086 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:01.147821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 487/967] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Aug 30 14:32:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:32:01.149119 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daae1f1-4609-42c6-8a1d-076b63646d08 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:01.271840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daae1f1-4609-42c6-8a1d-076b63646d08 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:01.309432 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1bc19-8c4f-48f8-b0c8-bc9f73e69dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:01.310058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1bc19-8c4f-48f8-b0c8-bc9f73e69dc6 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:01.310495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1bc19-8c4f-48f8-b0c8-bc9f73e69dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:01.310772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1bc19-8c4f-48f8-b0c8-bc9f73e69dc6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:01.311450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1bc19-8c4f-48f8-b0c8-bc9f73e69dc6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:01.312874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1bc19-8c4f-48f8-b0c8-bc9f73e69dc6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:01.357890 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 481/968] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:32:01 2022] POST /identity/v3/endpoints => generated 349 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:32:02.125472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475544e-bde2-4669-a3c2-e647ccaa7c54 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:02.126091 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475544e-bde2-4669-a3c2-e647ccaa7c54 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:02.126091 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475544e-bde2-4669-a3c2-e647ccaa7c54 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:02.127476 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 488/969] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:02.133386 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01108907-9299-412f-87eb-d9a08e8d01a3 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:02.133969 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01108907-9299-412f-87eb-d9a08e8d01a3 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:02.133969 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01108907-9299-412f-87eb-d9a08e8d01a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:02.156822 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-01108907-9299-412f-87eb-d9a08e8d01a3 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:02.158096 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-01108907-9299-412f-87eb-d9a08e8d01a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:02.230466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:02.277563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 482/970] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:02.564385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475544e-bde2-4669-a3c2-e647ccaa7c54 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:02.669204 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:02.700951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475544e-bde2-4669-a3c2-e647ccaa7c54 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:02.745774 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31bd29-8759-4d8a-800f-2b55245b12e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:02.746421 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31bd29-8759-4d8a-800f-2b55245b12e1 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:02.746699 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31bd29-8759-4d8a-800f-2b55245b12e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:02.746967 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31bd29-8759-4d8a-800f-2b55245b12e1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:02.759983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31bd29-8759-4d8a-800f-2b55245b12e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:32:02.766313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-bf31bd29-8759-4d8a-800f-2b55245b12e1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:32:02.766313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:02.766313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:02.766313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:02.766313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:02.766313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:02.766313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:02.766313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:02.766313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:02.766313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:02.766313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:02.766313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:02.766313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:32:02.766313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:32:02.766313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:32:02.766313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:32:02.767425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:32:02.767425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:32:02.767425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:32:02.767425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:32:02.767425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:02.767425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:02.767425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:02.767425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:02.767425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:02.767425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:02.767425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:02.767425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:02.767425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:02.767425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:02.767425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:02.767425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:02.772881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:02.772881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:02.772881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:02.772881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:02.772881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:32:02.772881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:32:02.772881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:32:02.772881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:32:02.772881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:32:02.772881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:32:02.772881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:32:02.772881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:32:02.772881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 489/971] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Aug 30 14:32:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:02.775181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01108907-9299-412f-87eb-d9a08e8d01a3 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:02.905787 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01108907-9299-412f-87eb-d9a08e8d01a3 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:02.949964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bd5c7-a0b4-46d7-b93a-b93fab2d5758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:02.950735 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bd5c7-a0b4-46d7-b93a-b93fab2d5758 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:02.951013 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bd5c7-a0b4-46d7-b93a-b93fab2d5758 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:02.951164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bd5c7-a0b4-46d7-b93a-b93fab2d5758 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:02.952286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bd5c7-a0b4-46d7-b93a-b93fab2d5758 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:02.952603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bd5c7-a0b4-46d7-b93a-b93fab2d5758 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:02.954015 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bd5c7-a0b4-46d7-b93a-b93fab2d5758 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:02.968514 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 483/972] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Aug 30 14:32:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:02.970403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31bd29-8759-4d8a-800f-2b55245b12e1 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:03.053035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492039ef-d744-4660-b310-942bcff82281 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:03.053342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492039ef-d744-4660-b310-942bcff82281 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:03.053578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492039ef-d744-4660-b310-942bcff82281 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:03.055102 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 484/973] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:03.060565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c8272-203c-43a5-8a7c-8b021c3d92c7 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:03.060565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c8272-203c-43a5-8a7c-8b021c3d92c7 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:03.061081 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c8272-203c-43a5-8a7c-8b021c3d92c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:03.079625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-678c8272-203c-43a5-8a7c-8b021c3d92c7 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:03.080888 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-678c8272-203c-43a5-8a7c-8b021c3d92c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:03.091418 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31bd29-8759-4d8a-800f-2b55245b12e1 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:03.122148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:03.141526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c92d46-7185-4876-b68a-f80ea643a428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:03.142678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c92d46-7185-4876-b68a-f80ea643a428 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:03.143082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c92d46-7185-4876-b68a-f80ea643a428 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:03.143739 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c92d46-7185-4876-b68a-f80ea643a428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:03.160491 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c92d46-7185-4876-b68a-f80ea643a428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:03.161904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 485/974] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:32:03.162525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c92d46-7185-4876-b68a-f80ea643a428 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:03.164288 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:03.180685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-40c92d46-7185-4876-b68a-f80ea643a428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:03.180685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:03.180685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:03.180685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:03.180685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:03.180685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:03.180685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:03.180685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:03.180685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:03.180685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:03.180685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:03.180685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:03.180685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:03.180685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:03.180685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:03.180685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:03.183188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:03.183188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:03.183188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:03.183188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:03.183188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:03.183188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:03.183188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:03.183188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:03.183188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:03.183188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:03.183188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:03.183188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:03.183188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:03.183188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:03.183188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:03.183188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:03.183188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:32:03.185444 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:03.185444 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:03.185444 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:32:03.185444 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 490/975] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:03.188369 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c8272-203c-43a5-8a7c-8b021c3d92c7 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:03.294784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c8272-203c-43a5-8a7c-8b021c3d92c7 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:03.331629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b160bb-19bf-41cb-aa4b-6d1a94c2b187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:03.332280 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b160bb-19bf-41cb-aa4b-6d1a94c2b187 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:03.332552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b160bb-19bf-41cb-aa4b-6d1a94c2b187 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:03.332813 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b160bb-19bf-41cb-aa4b-6d1a94c2b187 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:03.333470 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b160bb-19bf-41cb-aa4b-6d1a94c2b187 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:03.333762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b160bb-19bf-41cb-aa4b-6d1a94c2b187 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:03.335129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b160bb-19bf-41cb-aa4b-6d1a94c2b187 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:03.348209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 486/976] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:03.353383 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c92d46-7185-4876-b68a-f80ea643a428 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:03.384113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b382a-beb5-4e17-a2cf-15d825441d7c None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:03.384427 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b382a-beb5-4e17-a2cf-15d825441d7c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:03.384682 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b382a-beb5-4e17-a2cf-15d825441d7c None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:03.386054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 487/977] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:03.392438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db4036-350f-4f11-adcd-868d2d641c69 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:03.392769 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db4036-350f-4f11-adcd-868d2d641c69 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:03.392988 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db4036-350f-4f11-adcd-868d2d641c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:03.413905 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-83db4036-350f-4f11-adcd-868d2d641c69 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:03.415063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-83db4036-350f-4f11-adcd-868d2d641c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:03.451497 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:03.474754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c92d46-7185-4876-b68a-f80ea643a428 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:03.507877 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 488/978] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:32:03.508799 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db4036-350f-4f11-adcd-868d2d641c69 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:03.519206 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd639b-63db-46da-b671-5d0c13183bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:03.519861 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd639b-63db-46da-b671-5d0c13183bcf None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:03.520211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd639b-63db-46da-b671-5d0c13183bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:03.520441 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd639b-63db-46da-b671-5d0c13183bcf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:03.530654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd639b-63db-46da-b671-5d0c13183bcf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:03.532182 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd639b-63db-46da-b671-5d0c13183bcf None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:03.545918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-44fd639b-63db-46da-b671-5d0c13183bcf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:32:03.545918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:03.545918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:03.545918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:03.545918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:03.545918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:03.545918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:03.545918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:03.545918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:03.545918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:03.545918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:03.545918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:03.545918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:32:03.545918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:32:03.545918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:32:03.545918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:03.546988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:03.553049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:03.553049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:03.553049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:32:03.553049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:32:03.553049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:32:03.553049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:32:03.553049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:32:03.553049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:32:03.553049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:32:03.553049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:32:03.553049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 491/979] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Aug 30 14:32:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:03.553049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd639b-63db-46da-b671-5d0c13183bcf None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:03.623139 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:03.654907 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db4036-350f-4f11-adcd-868d2d641c69 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:03.684704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd639b-63db-46da-b671-5d0c13183bcf None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:03.694674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfec0d8-6163-4f33-829e-7a3c73ad3186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:03.695451 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfec0d8-6163-4f33-829e-7a3c73ad3186 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:03.695963 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfec0d8-6163-4f33-829e-7a3c73ad3186 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:03.696342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfec0d8-6163-4f33-829e-7a3c73ad3186 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:03.697129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfec0d8-6163-4f33-829e-7a3c73ad3186 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:03.698740 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfec0d8-6163-4f33-829e-7a3c73ad3186 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:03.710212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-1bfec0d8-6163-4f33-829e-7a3c73ad3186 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:32:03.710212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:03.710212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:32:03.710212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:32:03.710212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:32:03.710212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:32:03.710212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:32:03.710212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:32:03.710212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:32:03.710212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:03.710212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:32:03.710212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:03.710212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:03.710212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:03.710212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:03.710212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:03.710212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:03.711277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:03.711277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:03.711277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:03.711277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:03.711277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:03.711277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:03.711277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:32:03.711277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:32:03.711277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:32:03.711277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:32:03.711277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:03.711277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:03.711277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:03.711277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:03.711277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:03.711277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:03.712151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:03.712151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:03.712151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:03.712151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:03.712151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:03.712151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:03.712151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:03.712151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:03.712151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:03.712151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:03.712151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:32:03.712151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:32:03.712151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:32:03.712151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:03.712151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 489/980] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:32:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:03.718036 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfec0d8-6163-4f33-829e-7a3c73ad3186 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:03.726474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a6359-3a47-4796-a1fa-8fcac25f826f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:03.727095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a6359-3a47-4796-a1fa-8fcac25f826f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:03.727369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a6359-3a47-4796-a1fa-8fcac25f826f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:03.727629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a6359-3a47-4796-a1fa-8fcac25f826f None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:03.728425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a6359-3a47-4796-a1fa-8fcac25f826f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:03.728746 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a6359-3a47-4796-a1fa-8fcac25f826f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:03.730189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a6359-3a47-4796-a1fa-8fcac25f826f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:03.749225 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 492/981] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Aug 30 14:32:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:32:03.750274 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:03.756305 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a6359-3a47-4796-a1fa-8fcac25f826f None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:03.842876 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfec0d8-6163-4f33-829e-7a3c73ad3186 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:03.868500 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a6359-3a47-4796-a1fa-8fcac25f826f None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:03.880030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee79114-6b83-4abc-9c90-bea7a2249191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:03.880632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee79114-6b83-4abc-9c90-bea7a2249191 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:03.880908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee79114-6b83-4abc-9c90-bea7a2249191 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:03.881164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee79114-6b83-4abc-9c90-bea7a2249191 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:03.881791 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee79114-6b83-4abc-9c90-bea7a2249191 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:03.882081 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee79114-6b83-4abc-9c90-bea7a2249191 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:03.883247 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:03.883456 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee79114-6b83-4abc-9c90-bea7a2249191 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:03.889533 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 490/982] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Aug 30 14:32:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:32:03.890392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee79114-6b83-4abc-9c90-bea7a2249191 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:03.913303 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1eb19-400e-42d0-bd13-ba38ca08dfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:03.914348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1eb19-400e-42d0-bd13-ba38ca08dfba None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:03.914815 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1eb19-400e-42d0-bd13-ba38ca08dfba None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:03.915297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1eb19-400e-42d0-bd13-ba38ca08dfba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:03.993777 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:04.008340 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1eb19-400e-42d0-bd13-ba38ca08dfba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:04.009808 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1eb19-400e-42d0-bd13-ba38ca08dfba None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:04.014329 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee79114-6b83-4abc-9c90-bea7a2249191 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:04.066632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d5d9e-fe15-47c8-be38-70fe2c5126db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:04.067331 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d5d9e-fe15-47c8-be38-70fe2c5126db None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:04.067548 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d5d9e-fe15-47c8-be38-70fe2c5126db None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:04.067801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d5d9e-fe15-47c8-be38-70fe2c5126db None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:04.068415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d5d9e-fe15-47c8-be38-70fe2c5126db None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:04.069857 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d5d9e-fe15-47c8-be38-70fe2c5126db None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:04.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-7a3d5d9e-fe15-47c8-be38-70fe2c5126db None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:32:04.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:04.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:32:04.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:32:04.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:32:04.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:32:04.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:32:04.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:32:04.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:32:04.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:04.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:32:04.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:04.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:04.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:04.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:04.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:04.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:04.080866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:04.080866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:04.080866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:04.080866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:04.080866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:04.080866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:04.080866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:32:04.080866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:32:04.080866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:32:04.080866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:32:04.080866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:04.080866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:04.080866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:04.080866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:04.080866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:04.080866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:04.082082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:04.082082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:04.082082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:04.082082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:04.082082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:04.082082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:04.082082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:04.082082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:04.082082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:04.082082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:04.082082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:32:04.082082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:32:04.082082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:32:04.082082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:04.082082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 491/983] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Aug 30 14:32:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:04.082884 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d5d9e-fe15-47c8-be38-70fe2c5126db None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:04.124710 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 493/984] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:32:03 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 14:32:04.125587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1eb19-400e-42d0-bd13-ba38ca08dfba None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:04.141691 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:04.194744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d5d9e-fe15-47c8-be38-70fe2c5126db None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:04.231848 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4be8bf-57da-480c-b291-b86551e07e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:04.232544 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4be8bf-57da-480c-b291-b86551e07e96 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:04.232818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4be8bf-57da-480c-b291-b86551e07e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:04.233089 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4be8bf-57da-480c-b291-b86551e07e96 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:04.233724 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4be8bf-57da-480c-b291-b86551e07e96 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:04.234024 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4be8bf-57da-480c-b291-b86551e07e96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:04.235486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4be8bf-57da-480c-b291-b86551e07e96 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:04.235812 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1eb19-400e-42d0-bd13-ba38ca08dfba None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:04.240603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 492/985] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Aug 30 14:32:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:04.241424 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4be8bf-57da-480c-b291-b86551e07e96 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:04.267357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:04.276444 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d443f-e4f9-4477-a6d0-9b6bf6c8b7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:04.277084 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d443f-e4f9-4477-a6d0-9b6bf6c8b7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:04.277360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d443f-e4f9-4477-a6d0-9b6bf6c8b7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:04.277632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d443f-e4f9-4477-a6d0-9b6bf6c8b7f4 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:04.278270 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d443f-e4f9-4477-a6d0-9b6bf6c8b7f4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:04.278571 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d443f-e4f9-4477-a6d0-9b6bf6c8b7f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:04.280479 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d443f-e4f9-4477-a6d0-9b6bf6c8b7f4 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:04.285573 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 494/986] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Aug 30 14:32:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:04.286612 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d443f-e4f9-4477-a6d0-9b6bf6c8b7f4 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:04.357686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4be8bf-57da-480c-b291-b86551e07e96 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:04.374172 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:04.397008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9201444-8c0b-4085-9c83-71134fed6016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:04.397639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9201444-8c0b-4085-9c83-71134fed6016 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:04.397921 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9201444-8c0b-4085-9c83-71134fed6016 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:04.398189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9201444-8c0b-4085-9c83-71134fed6016 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:04.404713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d443f-e4f9-4477-a6d0-9b6bf6c8b7f4 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:04.410402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9201444-8c0b-4085-9c83-71134fed6016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:04.411865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9201444-8c0b-4085-9c83-71134fed6016 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:04.427447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-c9201444-8c0b-4085-9c83-71134fed6016 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:04.427447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:04.427447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:04.427447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:04.427447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:04.427447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:04.427447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:04.427447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:04.427447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:04.427447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:04.427447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:04.427447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:04.427447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:04.427447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:04.427447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:04.427447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:04.428930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:04.428930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:04.428930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:04.428930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:04.428930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:04.428930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:04.428930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:04.428930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:04.428930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:04.428930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:04.428930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:04.428930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:04.428930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:04.428930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:04.428930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:04.428930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:04.428930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:32:04.430681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:04.430681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:04.430681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:04.430681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 493/987] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:04.430681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9201444-8c0b-4085-9c83-71134fed6016 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:04.447301 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c310f77-14b7-44ac-85fd-39cae5c89dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:04.447925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c310f77-14b7-44ac-85fd-39cae5c89dac None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:04.448243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c310f77-14b7-44ac-85fd-39cae5c89dac None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:04.448510 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c310f77-14b7-44ac-85fd-39cae5c89dac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:04.449239 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c310f77-14b7-44ac-85fd-39cae5c89dac None admin] RBAC: Adding query filter params (interface=public, service_id=0b6d625468604382a823114c1272cc01, region_id=RegionOne) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:04.449542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c310f77-14b7-44ac-85fd-39cae5c89dac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:04.451719 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c310f77-14b7-44ac-85fd-39cae5c89dac None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:04.461162 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 495/988] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Aug 30 14:32:04 2022] GET /identity/v3/endpoints?service_id=0b6d625468604382a823114c1272cc01&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:04.462597 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c310f77-14b7-44ac-85fd-39cae5c89dac None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:04.509068 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:04.551974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9201444-8c0b-4085-9c83-71134fed6016 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:04.591049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c310f77-14b7-44ac-85fd-39cae5c89dac None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:04.605958 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737064c-6dfb-4f1a-bca1-df0b605f5fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:04.606977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737064c-6dfb-4f1a-bca1-df0b605f5fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:04.607472 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737064c-6dfb-4f1a-bca1-df0b605f5fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:04.607934 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737064c-6dfb-4f1a-bca1-df0b605f5fe6 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:04.609048 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737064c-6dfb-4f1a-bca1-df0b605f5fe6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:04.609575 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737064c-6dfb-4f1a-bca1-df0b605f5fe6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:04.612076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737064c-6dfb-4f1a-bca1-df0b605f5fe6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:04.625936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 494/989] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Aug 30 14:32:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:04.625936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737064c-6dfb-4f1a-bca1-df0b605f5fe6 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:04.635263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778880c-f638-411e-835a-7a4d47eaf82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:04.636288 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778880c-f638-411e-835a-7a4d47eaf82f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:04.636783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778880c-f638-411e-835a-7a4d47eaf82f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:04.637598 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778880c-f638-411e-835a-7a4d47eaf82f None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:04.639181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778880c-f638-411e-835a-7a4d47eaf82f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:04.639723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778880c-f638-411e-835a-7a4d47eaf82f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:04.642278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778880c-f638-411e-835a-7a4d47eaf82f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:04.657221 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:04.664169 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 496/990] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:04.665745 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778880c-f638-411e-835a-7a4d47eaf82f None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:04.735479 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737064c-6dfb-4f1a-bca1-df0b605f5fe6 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:04.782083 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:04.787897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fb84c-7715-446f-88bb-21529fc76184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:04.788555 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fb84c-7715-446f-88bb-21529fc76184 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:04.788659 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778880c-f638-411e-835a-7a4d47eaf82f None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:04.788833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fb84c-7715-446f-88bb-21529fc76184 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:04.789105 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fb84c-7715-446f-88bb-21529fc76184 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:04.789718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fb84c-7715-446f-88bb-21529fc76184 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:04.791106 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fb84c-7715-446f-88bb-21529fc76184 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:04.799010 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 495/991] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Aug 30 14:32:04 2022] GET /identity/v3/services => generated 1727 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:32:04.799851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fb84c-7715-446f-88bb-21529fc76184 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:04.851831 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89935a8-e2e5-424f-ac8e-f5b6f8efbcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:04.852790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89935a8-e2e5-424f-ac8e-f5b6f8efbcc7 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:04.853267 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89935a8-e2e5-424f-ac8e-f5b6f8efbcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:04.853751 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89935a8-e2e5-424f-ac8e-f5b6f8efbcc7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:04.855125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89935a8-e2e5-424f-ac8e-f5b6f8efbcc7 None admin] RBAC: Adding query filter params (interface=admin, service_id=dd0695a6299a44d8a4c1433fedfd1e28, region_id=RegionOne) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:04.855677 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89935a8-e2e5-424f-ac8e-f5b6f8efbcc7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:04.858167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89935a8-e2e5-424f-ac8e-f5b6f8efbcc7 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:04.873291 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 497/992] 10.222.0.25 () {56 vars in 1249 bytes} [Tue Aug 30 14:32:04 2022] GET /identity/v3/endpoints?service_id=dd0695a6299a44d8a4c1433fedfd1e28&interface=admin®ion_id=RegionOne => generated 193 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:04.882364 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89935a8-e2e5-424f-ac8e-f5b6f8efbcc7 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:04.948159 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fb84c-7715-446f-88bb-21529fc76184 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:04.981109 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:04.991894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abb3c8-8fc5-48e0-be4d-fe400caedf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:04.992568 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abb3c8-8fc5-48e0-be4d-fe400caedf38 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:04.992850 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abb3c8-8fc5-48e0-be4d-fe400caedf38 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:04.993115 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abb3c8-8fc5-48e0-be4d-fe400caedf38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:05.002894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03abb3c8-8fc5-48e0-be4d-fe400caedf38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:05.004733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03abb3c8-8fc5-48e0-be4d-fe400caedf38 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:05.017355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-03abb3c8-8fc5-48e0-be4d-fe400caedf38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:32:05.017355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:05.017355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:05.017355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:05.017355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:05.017355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:05.017355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:05.017355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:05.017355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:05.017355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:05.017355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:05.017355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:05.017355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:32:05.017355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:32:05.017355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:32:05.021903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:32:05.021903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:05.021903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:05.021903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:05.021903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:05.021903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:05.021903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:05.021903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:05.021903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:05.021903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:05.021903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:05.021903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:05.021903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:05.021903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:05.021903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:05.021903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:05.021903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:05.030707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:32:05.030707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:32:05.030707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:32:05.030707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:32:05.030707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:32:05.030707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:32:05.030707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:32:05.030707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:05.030707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 496/993] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:32:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:05.030707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abb3c8-8fc5-48e0-be4d-fe400caedf38 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:05.045530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89935a8-e2e5-424f-ac8e-f5b6f8efbcc7 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:05.106734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4a7d5-0d1b-424a-a1d1-bd5155d59a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:05.106734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4a7d5-0d1b-424a-a1d1-bd5155d59a05 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:05.106734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4a7d5-0d1b-424a-a1d1-bd5155d59a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:05.106734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4a7d5-0d1b-424a-a1d1-bd5155d59a05 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:05.107420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b4a7d5-0d1b-424a-a1d1-bd5155d59a05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:05.108268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b4a7d5-0d1b-424a-a1d1-bd5155d59a05 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:05.108404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:05.115734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 498/994] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Aug 30 14:32:04 2022] GET /identity/v3/services => generated 1727 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:32:05.169625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abb3c8-8fc5-48e0-be4d-fe400caedf38 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:05.219037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230288c9-c9b1-4b06-a6ad-aa2bbe470edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:05.220534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230288c9-c9b1-4b06-a6ad-aa2bbe470edf None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:05.220983 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230288c9-c9b1-4b06-a6ad-aa2bbe470edf None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:05.221584 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230288c9-c9b1-4b06-a6ad-aa2bbe470edf None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:05.222615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230288c9-c9b1-4b06-a6ad-aa2bbe470edf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:05.223096 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230288c9-c9b1-4b06-a6ad-aa2bbe470edf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:05.225111 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230288c9-c9b1-4b06-a6ad-aa2bbe470edf None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:05.241960 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 497/995] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:32:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:32:05.250508 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4a7d5-0d1b-424a-a1d1-bd5155d59a05 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:05.408362 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4a7d5-0d1b-424a-a1d1-bd5155d59a05 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:05.454719 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:05.472596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83922bfe-00d1-487c-a26c-8b7da1b60ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:05.473227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83922bfe-00d1-487c-a26c-8b7da1b60ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:05.473500 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83922bfe-00d1-487c-a26c-8b7da1b60ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:05.473766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83922bfe-00d1-487c-a26c-8b7da1b60ce0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:05.474511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83922bfe-00d1-487c-a26c-8b7da1b60ce0 None admin] RBAC: Adding query filter params (role.id=2295f599ff36442faecde157de2c08c8, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=8fc940594a1848aa9644b645049a1bd6) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:05.474812 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83922bfe-00d1-487c-a26c-8b7da1b60ce0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:05.476286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83922bfe-00d1-487c-a26c-8b7da1b60ce0 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:05.483067 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 499/996] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:32:05 2022] GET /identity/v3/role_assignments?user.id=8fc940594a1848aa9644b645049a1bd6&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=2295f599ff36442faecde157de2c08c8 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:06.827763 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfa523-b6e6-41e2-bd57-261e9d448936 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:06.828361 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfa523-b6e6-41e2-bd57-261e9d448936 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:06.828361 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfa523-b6e6-41e2-bd57-261e9d448936 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:06.832050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 498/997] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:06.835543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc4857-b63b-49ce-aa2e-e04549150d18 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:06.835977 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc4857-b63b-49ce-aa2e-e04549150d18 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:06.836190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc4857-b63b-49ce-aa2e-e04549150d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:06.855839 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-cbfc4857-b63b-49ce-aa2e-e04549150d18 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:06.857114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-cbfc4857-b63b-49ce-aa2e-e04549150d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:06.932120 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 500/998] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:07.097338 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b1e80-e67d-4e0f-843c-325d8c5e214b None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:07.097714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b1e80-e67d-4e0f-843c-325d8c5e214b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:07.097876 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b1e80-e67d-4e0f-843c-325d8c5e214b None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:07.099419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 499/999] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:07.104653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdac49-99ea-41ac-a2b1-d09c9d986c57 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:07.104925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdac49-99ea-41ac-a2b1-d09c9d986c57 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:07.105205 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdac49-99ea-41ac-a2b1-d09c9d986c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:07.123917 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-8ebdac49-99ea-41ac-a2b1-d09c9d986c57 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:07.125138 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-8ebdac49-99ea-41ac-a2b1-d09c9d986c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:07.197250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 501/1000] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:07 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:07.214282 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b1e80-e67d-4e0f-843c-325d8c5e214b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:07.323396 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b1e80-e67d-4e0f-843c-325d8c5e214b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:07.342714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:07.362735 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb02bf-ea03-4721-b9a9-bc017920e0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:07.363379 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb02bf-ea03-4721-b9a9-bc017920e0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:07.363658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb02bf-ea03-4721-b9a9-bc017920e0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:07.364051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb02bf-ea03-4721-b9a9-bc017920e0a9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:07.364600 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb02bf-ea03-4721-b9a9-bc017920e0a9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:07.366068 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb02bf-ea03-4721-b9a9-bc017920e0a9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:07.375290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-ecbb02bf-ea03-4721-b9a9-bc017920e0a9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:32:07.375290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:07.375290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:32:07.375290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:32:07.375290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:32:07.375290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:32:07.375290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:32:07.375290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:32:07.375290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:32:07.375290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:07.375290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:32:07.375290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:07.375290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:07.375290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:07.375290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:07.375290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:07.375290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:07.380321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:07.380321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:07.380321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:07.380321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:07.380321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:07.380321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:07.380321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:32:07.380321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:32:07.380321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:32:07.380321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:32:07.380321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:07.380321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:07.380321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:07.380321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:07.380321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:07.380321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:07.381522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:07.381522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:07.381522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:07.381522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:07.381522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:07.381522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:07.381522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:07.381522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:07.381522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:07.381522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:07.381522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:32:07.381522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:32:07.381522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:32:07.381522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:07.381522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 500/1001] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:32:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:07.382742 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdac49-99ea-41ac-a2b1-d09c9d986c57 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:07.440226 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:07.462522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb02bf-ea03-4721-b9a9-bc017920e0a9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:07.492303 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdac49-99ea-41ac-a2b1-d09c9d986c57 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:07.526718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096a0ff-5ee5-44c0-8cef-3d90e5280455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:07.527338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096a0ff-5ee5-44c0-8cef-3d90e5280455 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:07.527616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096a0ff-5ee5-44c0-8cef-3d90e5280455 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:07.527888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096a0ff-5ee5-44c0-8cef-3d90e5280455 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:07.528558 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096a0ff-5ee5-44c0-8cef-3d90e5280455 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:07.528883 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096a0ff-5ee5-44c0-8cef-3d90e5280455 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:07.530352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096a0ff-5ee5-44c0-8cef-3d90e5280455 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:07.535110 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 502/1002] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Aug 30 14:32:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:07.536409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6422b-981e-437a-bba5-80465a42f036 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:07.536698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6422b-981e-437a-bba5-80465a42f036 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:07.536972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6422b-981e-437a-bba5-80465a42f036 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:07.538335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 503/1003] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:07.540777 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6422b-981e-437a-bba5-80465a42f036 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:07.565402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb02bf-ea03-4721-b9a9-bc017920e0a9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:07.600341 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b907f8-77c4-4fe3-b3bc-3f8d9194a61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:07.600982 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b907f8-77c4-4fe3-b3bc-3f8d9194a61f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:07.601259 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b907f8-77c4-4fe3-b3bc-3f8d9194a61f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:07.601534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b907f8-77c4-4fe3-b3bc-3f8d9194a61f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:07.602122 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b907f8-77c4-4fe3-b3bc-3f8d9194a61f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:07.603500 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b907f8-77c4-4fe3-b3bc-3f8d9194a61f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:07.612335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-91b907f8-77c4-4fe3-b3bc-3f8d9194a61f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:32:07.612335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:07.612335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:32:07.612335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:32:07.612335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:32:07.612335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:32:07.612335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:32:07.612335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:32:07.612335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:32:07.612335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:07.612335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:32:07.612335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:07.612335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:07.612335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:07.612335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:07.612335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:07.612335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:07.613553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:07.613553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:07.613553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:07.613553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:07.613553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:07.613553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:07.613553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:32:07.613553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:32:07.613553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:32:07.613553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:32:07.613553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:07.613553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:07.613553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:07.613553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:07.613553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:07.613553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:07.614565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:07.614565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:07.614565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:07.614565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:07.614565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:07.614565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:07.614565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:07.614565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:07.614565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:07.614565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:07.614565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:32:07.614565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:32:07.614565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:32:07.614565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:07.614565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 501/1004] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Aug 30 14:32:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:32:07.615536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e9a56-7330-411f-b66a-29f335882c53 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:07.615536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e9a56-7330-411f-b66a-29f335882c53 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:07.615687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e9a56-7330-411f-b66a-29f335882c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:07.634622 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f76e9a56-7330-411f-b66a-29f335882c53 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:07.635831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f76e9a56-7330-411f-b66a-29f335882c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:07.649217 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6422b-981e-437a-bba5-80465a42f036 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:07.666455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:07.677168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:07.685595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b1a29-0dcb-4625-be53-be9cc8a34f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:07.686207 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b1a29-0dcb-4625-be53-be9cc8a34f40 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:07.686479 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b1a29-0dcb-4625-be53-be9cc8a34f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:07.686743 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b1a29-0dcb-4625-be53-be9cc8a34f40 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:07.687373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b1a29-0dcb-4625-be53-be9cc8a34f40 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:07.687668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b1a29-0dcb-4625-be53-be9cc8a34f40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:07.689110 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b1a29-0dcb-4625-be53-be9cc8a34f40 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:07.694047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 504/1005] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Aug 30 14:32:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:32:07.694829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b1a29-0dcb-4625-be53-be9cc8a34f40 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:07.711522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 502/1006] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:07 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:07.712876 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e9a56-7330-411f-b66a-29f335882c53 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:07.810783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b1a29-0dcb-4625-be53-be9cc8a34f40 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:07.815014 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e9a56-7330-411f-b66a-29f335882c53 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:07.851616 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906f24e-78ac-457a-b69a-47c058b0e60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:07.852308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2f67f-0a85-4735-aac5-87ee6cdd406b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:07.852704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2f67f-0a85-4735-aac5-87ee6cdd406b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:07.852920 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2f67f-0a85-4735-aac5-87ee6cdd406b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:07.853191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2f67f-0a85-4735-aac5-87ee6cdd406b None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:07.853727 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906f24e-78ac-457a-b69a-47c058b0e60c None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:07.853837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d2f67f-0a85-4735-aac5-87ee6cdd406b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:07.854112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d2f67f-0a85-4735-aac5-87ee6cdd406b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:07.854382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906f24e-78ac-457a-b69a-47c058b0e60c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:07.854778 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906f24e-78ac-457a-b69a-47c058b0e60c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:07.855485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d2f67f-0a85-4735-aac5-87ee6cdd406b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:07.855729 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906f24e-78ac-457a-b69a-47c058b0e60c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:07.857693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906f24e-78ac-457a-b69a-47c058b0e60c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:07.864160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 505/1007] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Aug 30 14:32:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:07.866721 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2f67f-0a85-4735-aac5-87ee6cdd406b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:07.888052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 503/1008] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:32:07 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:32:07.918369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:07.975228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906f24e-78ac-457a-b69a-47c058b0e60c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:07.982706 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2f67f-0a85-4735-aac5-87ee6cdd406b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:08.011619 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:08.032105 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb042c-f641-417b-812b-0bd6c86d609e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:08.032793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb042c-f641-417b-812b-0bd6c86d609e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:08.033058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb042c-f641-417b-812b-0bd6c86d609e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:08.033980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb042c-f641-417b-812b-0bd6c86d609e None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:08.033980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb042c-f641-417b-812b-0bd6c86d609e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:08.034227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb042c-f641-417b-812b-0bd6c86d609e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:08.036690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb042c-f641-417b-812b-0bd6c86d609e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:08.041422 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 506/1009] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Aug 30 14:32:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:32:08.047830 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb042c-f641-417b-812b-0bd6c86d609e None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:08.105332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906f24e-78ac-457a-b69a-47c058b0e60c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:08.145989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f177b7d-d583-4126-abb1-93ea14a37863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:08.146662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f177b7d-d583-4126-abb1-93ea14a37863 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:08.146938 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f177b7d-d583-4126-abb1-93ea14a37863 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:08.147205 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f177b7d-d583-4126-abb1-93ea14a37863 None admin] PATH_INFO: `/v3/services/image` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:08.147786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f177b7d-d583-4126-abb1-93ea14a37863 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:08.149211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f177b7d-d583-4126-abb1-93ea14a37863 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:08.164761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-7f177b7d-d583-4126-abb1-93ea14a37863 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:32:08.164761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:08.164761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:32:08.164761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:32:08.164761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:32:08.164761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:32:08.164761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:32:08.164761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:32:08.164761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:32:08.164761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:08.164761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:32:08.164761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:08.164761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:08.164761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:08.164761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:08.164761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:08.164761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:08.165849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:08.165849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:08.165849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:08.165849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:08.165849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:08.165849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:08.165849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:32:08.165849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:32:08.165849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:32:08.165849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:32:08.165849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:08.165849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:08.165849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:08.165849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:08.165849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:08.165849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:08.166699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:08.166699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:08.166699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:08.166699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:08.166699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:08.166699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:08.166699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:08.166699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:08.166699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:08.166699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:08.166699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:32:08.166699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:32:08.166699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:32:08.166699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:08.169003 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 504/1010] 10.222.0.25 () {56 vars in 1105 bytes} [Tue Aug 30 14:32:07 2022] GET /identity/v3/services/image => generated 86 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:32:08.175787 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f177b7d-d583-4126-abb1-93ea14a37863 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:08.190996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb042c-f641-417b-812b-0bd6c86d609e None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:08.194924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:08.234618 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25908e0c-7758-4d89-8287-386cc0051e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:08.235147 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25908e0c-7758-4d89-8287-386cc0051e07 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:08.235423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25908e0c-7758-4d89-8287-386cc0051e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:08.235691 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25908e0c-7758-4d89-8287-386cc0051e07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:08.236396 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25908e0c-7758-4d89-8287-386cc0051e07 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:08.238243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25908e0c-7758-4d89-8287-386cc0051e07 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:08.268027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 507/1011] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:32:08 2022] POST /identity/v3/endpoints => generated 325 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:32:08.339198 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:08.353651 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f177b7d-d583-4126-abb1-93ea14a37863 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:08.426762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09914340-c0d1-4c5c-9f0c-9d0b51968365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:08.427352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09914340-c0d1-4c5c-9f0c-9d0b51968365 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:08.427647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09914340-c0d1-4c5c-9f0c-9d0b51968365 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:08.427896 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09914340-c0d1-4c5c-9f0c-9d0b51968365 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:08.428524 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09914340-c0d1-4c5c-9f0c-9d0b51968365 None admin] RBAC: Adding query filter params (name=image) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:08.428817 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09914340-c0d1-4c5c-9f0c-9d0b51968365 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:08.430188 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09914340-c0d1-4c5c-9f0c-9d0b51968365 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:08.435864 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 505/1012] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:08.441832 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25908e0c-7758-4d89-8287-386cc0051e07 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:08.552585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25908e0c-7758-4d89-8287-386cc0051e07 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:08.595173 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375291a-65f6-4428-8177-74dc495b8399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:08.595828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375291a-65f6-4428-8177-74dc495b8399 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:08.596126 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375291a-65f6-4428-8177-74dc495b8399 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:08.596359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375291a-65f6-4428-8177-74dc495b8399 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:08.597004 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375291a-65f6-4428-8177-74dc495b8399 None admin] RBAC: Adding query filter params (type=image) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:08.597305 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375291a-65f6-4428-8177-74dc495b8399 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:08.598672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375291a-65f6-4428-8177-74dc495b8399 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:08.603864 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 508/1013] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:08 2022] GET /identity/v3/services?type=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:09.911443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67040a5b-861b-4b54-9572-0242aaf36de3 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:09.911949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67040a5b-861b-4b54-9572-0242aaf36de3 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:09.911949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67040a5b-861b-4b54-9572-0242aaf36de3 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:09.913362 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 506/1014] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:09.918878 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5fb6f-3fa4-4065-99ff-437c9d832fdf None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:09.919334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5fb6f-3fa4-4065-99ff-437c9d832fdf None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:09.919465 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5fb6f-3fa4-4065-99ff-437c9d832fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:09.939160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-dca5fb6f-3fa4-4065-99ff-437c9d832fdf None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:09.940362 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-dca5fb6f-3fa4-4065-99ff-437c9d832fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:09.965600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:10.014448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 509/1015] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:09 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:10.232743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64222ef0-8a6f-4ba5-99aa-bd4d59cf3928 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:10.233100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64222ef0-8a6f-4ba5-99aa-bd4d59cf3928 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:10.233291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64222ef0-8a6f-4ba5-99aa-bd4d59cf3928 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:10.234781 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 507/1016] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:10.240593 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00321cd-fd31-436e-a9be-2e8eb4a2540c None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:10.240905 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00321cd-fd31-436e-a9be-2e8eb4a2540c None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:10.241110 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00321cd-fd31-436e-a9be-2e8eb4a2540c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:10.259937 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-e00321cd-fd31-436e-a9be-2e8eb4a2540c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:10.261143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e00321cd-fd31-436e-a9be-2e8eb4a2540c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:10.282552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64222ef0-8a6f-4ba5-99aa-bd4d59cf3928 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:10.337860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 510/1017] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:10 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:10.387916 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64222ef0-8a6f-4ba5-99aa-bd4d59cf3928 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:10.428285 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c81d46-821c-42e1-a09f-70f0ab4395e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:10.428923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c81d46-821c-42e1-a09f-70f0ab4395e3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:10.429177 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c81d46-821c-42e1-a09f-70f0ab4395e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:10.429482 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c81d46-821c-42e1-a09f-70f0ab4395e3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:10.442141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c81d46-821c-42e1-a09f-70f0ab4395e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:32:10.446977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-27c81d46-821c-42e1-a09f-70f0ab4395e3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:32:10.446977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:10.446977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:10.446977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:10.446977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:10.446977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:10.446977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:10.446977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:10.446977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:10.446977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:10.446977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:10.446977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:10.446977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:32:10.446977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:32:10.446977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:32:10.446977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:32:10.447910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:32:10.447910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:32:10.447910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:32:10.447910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:32:10.447910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:10.447910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:10.447910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:10.447910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:10.447910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:10.447910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:10.447910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:10.447910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:10.447910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:10.447910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:10.447910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:10.447910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:10.449103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:10.449103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:10.449103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:10.449103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:10.449103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:32:10.449103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:32:10.449103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:32:10.449103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:32:10.449103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:32:10.449103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:32:10.449103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:32:10.449103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:10.449103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 508/1018] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Aug 30 14:32:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:10.455229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00321cd-fd31-436e-a9be-2e8eb4a2540c None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:10.547091 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:10.561666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00321cd-fd31-436e-a9be-2e8eb4a2540c None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:10.592907 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fcccc-083a-4ffe-be29-903451146d7c None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:10.593208 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fcccc-083a-4ffe-be29-903451146d7c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:10.593445 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fcccc-083a-4ffe-be29-903451146d7c None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:10.594911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 509/1019] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:32:10.599925 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61008e17-9b22-4634-8100-f122b2ea281d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:10.600262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61008e17-9b22-4634-8100-f122b2ea281d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:10.600522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61008e17-9b22-4634-8100-f122b2ea281d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:10.604318 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043c0cd-bb72-4e7b-aa9f-a431502d7cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:10.604993 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043c0cd-bb72-4e7b-aa9f-a431502d7cba None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:10.605220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043c0cd-bb72-4e7b-aa9f-a431502d7cba None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:10.605488 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043c0cd-bb72-4e7b-aa9f-a431502d7cba None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:10.606161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043c0cd-bb72-4e7b-aa9f-a431502d7cba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:10.606461 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043c0cd-bb72-4e7b-aa9f-a431502d7cba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:10.608696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043c0cd-bb72-4e7b-aa9f-a431502d7cba None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:10.617775 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 511/1020] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Aug 30 14:32:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:10.619467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-61008e17-9b22-4634-8100-f122b2ea281d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:10.619735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043c0cd-bb72-4e7b-aa9f-a431502d7cba None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:10.620699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-61008e17-9b22-4634-8100-f122b2ea281d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:10.633795 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:10.693674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 510/1021] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:10 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:32:10.694907 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61008e17-9b22-4634-8100-f122b2ea281d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:10.726985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043c0cd-bb72-4e7b-aa9f-a431502d7cba None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:10.762202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd2f02-5f66-4320-8905-4959d7acdd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:10.762873 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd2f02-5f66-4320-8905-4959d7acdd2a None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:10.763141 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd2f02-5f66-4320-8905-4959d7acdd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:10.763511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd2f02-5f66-4320-8905-4959d7acdd2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:10.764653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd2f02-5f66-4320-8905-4959d7acdd2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:10.766082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd2f02-5f66-4320-8905-4959d7acdd2a None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:10.767958 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 512/1022] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:10.773482 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd2f02-5f66-4320-8905-4959d7acdd2a None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:10.798691 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:10.803013 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61008e17-9b22-4634-8100-f122b2ea281d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:10.844827 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88b744-e57f-43c9-b205-ca60c838d7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:10.845468 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88b744-e57f-43c9-b205-ca60c838d7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:10.845747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88b744-e57f-43c9-b205-ca60c838d7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:10.846018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88b744-e57f-43c9-b205-ca60c838d7b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:10.856544 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88b744-e57f-43c9-b205-ca60c838d7b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:10.858233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88b744-e57f-43c9-b205-ca60c838d7b7 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:10.861649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:10.878423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-2e88b744-e57f-43c9-b205-ca60c838d7b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:10.878423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:10.878423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:10.878423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:10.878423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:10.878423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:10.878423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:10.878423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:10.878423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:10.878423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:10.878423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:10.878423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:10.878423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:10.878423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:10.878423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:10.878423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:10.879646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:10.879646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:10.879646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:10.879646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:10.879646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:10.879646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:10.879646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:10.879646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:10.879646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:10.879646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:10.879646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:10.879646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:10.879646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:10.879646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:10.879646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:10.879646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:10.879646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:32:10.880602 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:10.880602 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:10.880602 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:10.880602 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 511/1023] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:10.891043 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88b744-e57f-43c9-b205-ca60c838d7b7 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:10.926321 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd2f02-5f66-4320-8905-4959d7acdd2a None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:10.972719 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9d6fb-b08a-4e31-b319-0a41a53fee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:10.973344 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9d6fb-b08a-4e31-b319-0a41a53fee32 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:10.973622 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9d6fb-b08a-4e31-b319-0a41a53fee32 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:10.973897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9d6fb-b08a-4e31-b319-0a41a53fee32 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:10.974559 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9d6fb-b08a-4e31-b319-0a41a53fee32 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:10.975991 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9d6fb-b08a-4e31-b319-0a41a53fee32 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:11.005618 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 513/1024] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Aug 30 14:32:10 2022] POST /identity/v3/projects => generated 311 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:32:11.006442 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9d6fb-b08a-4e31-b319-0a41a53fee32 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:11.015410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88b744-e57f-43c9-b205-ca60c838d7b7 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:11.061884 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382cd98-f8fd-4c94-a20a-0cb1d743e369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:11.063554 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382cd98-f8fd-4c94-a20a-0cb1d743e369 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:11.064083 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382cd98-f8fd-4c94-a20a-0cb1d743e369 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:11.064530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382cd98-f8fd-4c94-a20a-0cb1d743e369 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:11.065878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382cd98-f8fd-4c94-a20a-0cb1d743e369 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:11.066350 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382cd98-f8fd-4c94-a20a-0cb1d743e369 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:11.068660 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382cd98-f8fd-4c94-a20a-0cb1d743e369 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:11.089675 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 512/1025] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:11.097897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382cd98-f8fd-4c94-a20a-0cb1d743e369 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:11.104191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:11.150009 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9d6fb-b08a-4e31-b319-0a41a53fee32 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:11.170739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:11.201635 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f33127-45ad-4b9e-8243-f62099f4a981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:11.202261 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f33127-45ad-4b9e-8243-f62099f4a981 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:11.202545 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f33127-45ad-4b9e-8243-f62099f4a981 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:11.202815 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f33127-45ad-4b9e-8243-f62099f4a981 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:11.203475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f33127-45ad-4b9e-8243-f62099f4a981 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:11.204892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f33127-45ad-4b9e-8243-f62099f4a981 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:11.223254 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 514/1026] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Aug 30 14:32:11 2022] POST /identity/v3/services => generated 242 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:32:11.226302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382cd98-f8fd-4c94-a20a-0cb1d743e369 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:11.274912 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb7775-9fa6-4203-8ce4-dc4740b42479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:11.275636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb7775-9fa6-4203-8ce4-dc4740b42479 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:11.275912 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb7775-9fa6-4203-8ce4-dc4740b42479 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:11.276204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb7775-9fa6-4203-8ce4-dc4740b42479 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:11.284949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb7775-9fa6-4203-8ce4-dc4740b42479 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:11.286432 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb7775-9fa6-4203-8ce4-dc4740b42479 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:11.300504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-e4fb7775-9fa6-4203-8ce4-dc4740b42479 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:32:11.300504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:11.300504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:11.300504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:11.300504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:11.300504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:11.300504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:11.300504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:11.300504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:11.300504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:11.300504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:11.300504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:11.300504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:32:11.300504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:32:11.300504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:32:11.300504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:11.301658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:11.302669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:11.302669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:11.302669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:32:11.302669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:32:11.302669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:32:11.302669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:32:11.302669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:32:11.302669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:32:11.302669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:32:11.302669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:11.302669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 513/1027] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Aug 30 14:32:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:11.307981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f33127-45ad-4b9e-8243-f62099f4a981 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:11.432915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f33127-45ad-4b9e-8243-f62099f4a981 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:11.463572 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:11.472047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49080c8-9c0c-4d07-8df3-f80111f171a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:11.472875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49080c8-9c0c-4d07-8df3-f80111f171a4 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:11.473060 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49080c8-9c0c-4d07-8df3-f80111f171a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:11.473417 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49080c8-9c0c-4d07-8df3-f80111f171a4 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:11.474240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49080c8-9c0c-4d07-8df3-f80111f171a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:11.474616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49080c8-9c0c-4d07-8df3-f80111f171a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:11.476167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49080c8-9c0c-4d07-8df3-f80111f171a4 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:11.498351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 515/1028] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Aug 30 14:32:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:11.504780 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb7775-9fa6-4203-8ce4-dc4740b42479 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:11.629819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb7775-9fa6-4203-8ce4-dc4740b42479 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:11.650878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:11.686892 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef2b3c-deaf-48f2-b8fd-8f831f4f01c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:11.687889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef2b3c-deaf-48f2-b8fd-8f831f4f01c4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:11.688323 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef2b3c-deaf-48f2-b8fd-8f831f4f01c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:11.688778 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef2b3c-deaf-48f2-b8fd-8f831f4f01c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:11.808651 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef2b3c-deaf-48f2-b8fd-8f831f4f01c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:11.811324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef2b3c-deaf-48f2-b8fd-8f831f4f01c4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:11.930954 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 514/1029] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:32:11 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:32:11.932373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49080c8-9c0c-4d07-8df3-f80111f171a4 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:12.042963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49080c8-9c0c-4d07-8df3-f80111f171a4 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:12.081933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef0b6a-d8d5-42a4-8232-57af902461ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:12.082634 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef0b6a-d8d5-42a4-8232-57af902461ab None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:12.082914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef0b6a-d8d5-42a4-8232-57af902461ab None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:12.083166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef0b6a-d8d5-42a4-8232-57af902461ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:12.094666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef0b6a-d8d5-42a4-8232-57af902461ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:12.096033 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef0b6a-d8d5-42a4-8232-57af902461ab None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:12.112330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-e3ef0b6a-d8d5-42a4-8232-57af902461ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:12.112330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:12.112330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:12.112330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:12.112330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:12.112330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:12.112330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:12.112330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:12.112330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:12.112330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:12.112330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:12.112330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:12.112330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:12.112330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:12.112330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:12.112330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:12.119111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:12.119111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:12.119111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:12.119111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:12.119111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:12.119111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:12.119111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:12.119111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:12.119111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:12.119111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:12.119111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:12.119111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:12.119111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:12.119111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:12.119111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:12.119111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:12.119111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:32:12.120110 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:12.120110 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:12.120110 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:32:12.120110 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 516/1030] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:12.121543 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef2b3c-deaf-48f2-b8fd-8f831f4f01c4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:12.146221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:12.256035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef2b3c-deaf-48f2-b8fd-8f831f4f01c4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:12.293277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4bf39-c1cc-4e18-9d1d-fe2e21980d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:12.293884 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4bf39-c1cc-4e18-9d1d-fe2e21980d18 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:12.294072 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4bf39-c1cc-4e18-9d1d-fe2e21980d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:12.294620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4bf39-c1cc-4e18-9d1d-fe2e21980d18 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:12.295847 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4bf39-c1cc-4e18-9d1d-fe2e21980d18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:12.296193 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4bf39-c1cc-4e18-9d1d-fe2e21980d18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:12.297593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4bf39-c1cc-4e18-9d1d-fe2e21980d18 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:12.310046 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 515/1031] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:12.316653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef0b6a-d8d5-42a4-8232-57af902461ab None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:12.365937 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:12.436168 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef0b6a-d8d5-42a4-8232-57af902461ab None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:12.474601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2fb51-c3e8-4ad8-af88-12c55c8939b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:12.475238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2fb51-c3e8-4ad8-af88-12c55c8939b7 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:12.475509 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2fb51-c3e8-4ad8-af88-12c55c8939b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:12.475775 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2fb51-c3e8-4ad8-af88-12c55c8939b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:12.483288 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2fb51-c3e8-4ad8-af88-12c55c8939b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:12.484775 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2fb51-c3e8-4ad8-af88-12c55c8939b7 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:12.496576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-8eb2fb51-c3e8-4ad8-af88-12c55c8939b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:32:12.496576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:12.496576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:12.496576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:12.496576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:12.496576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:12.496576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:12.496576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:12.496576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:12.496576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:12.496576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:12.496576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:12.496576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:32:12.496576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:32:12.496576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:32:12.497709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:32:12.497709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:12.497709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:12.497709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:12.497709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:12.497709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:12.497709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:12.497709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:12.497709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:12.497709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:12.497709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:12.497709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:12.497709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:12.497709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:12.497709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:12.497709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:12.497709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:12.504229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:32:12.504229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:32:12.504229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:32:12.504229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:32:12.504229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:32:12.504229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:32:12.504229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:32:12.504229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:32:12.504229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 517/1032] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:32:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:12.505008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4bf39-c1cc-4e18-9d1d-fe2e21980d18 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:12.591489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:12.632655 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4bf39-c1cc-4e18-9d1d-fe2e21980d18 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:12.672057 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409dc6a-83ba-4ec8-9d9e-83ed9e579712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:12.672719 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409dc6a-83ba-4ec8-9d9e-83ed9e579712 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:12.672951 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409dc6a-83ba-4ec8-9d9e-83ed9e579712 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:12.673260 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409dc6a-83ba-4ec8-9d9e-83ed9e579712 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:12.673967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409dc6a-83ba-4ec8-9d9e-83ed9e579712 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:12.674269 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409dc6a-83ba-4ec8-9d9e-83ed9e579712 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:12.675718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409dc6a-83ba-4ec8-9d9e-83ed9e579712 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:12.690024 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 516/1033] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:32:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:12.696083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2fb51-c3e8-4ad8-af88-12c55c8939b7 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:12.803977 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:12.807773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2fb51-c3e8-4ad8-af88-12c55c8939b7 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:12.846175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169eb24-53f2-4872-b4fd-d5096edec0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:12.846696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169eb24-53f2-4872-b4fd-d5096edec0ae None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:12.846976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169eb24-53f2-4872-b4fd-d5096edec0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:12.847230 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169eb24-53f2-4872-b4fd-d5096edec0ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:12.847983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169eb24-53f2-4872-b4fd-d5096edec0ae None admin] RBAC: Adding query filter params (role.id=2295f599ff36442faecde157de2c08c8, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=cc95bbdafb0c4f34ad0662bc2e3ba4ec) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:12.848315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169eb24-53f2-4872-b4fd-d5096edec0ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:12.849745 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169eb24-53f2-4872-b4fd-d5096edec0ae None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:12.855900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 518/1034] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:32:12 2022] GET /identity/v3/role_assignments?user.id=cc95bbdafb0c4f34ad0662bc2e3ba4ec&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=2295f599ff36442faecde157de2c08c8 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:12.988543 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cbea0-5888-45ef-b809-ab12ba2601df None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:12.988904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cbea0-5888-45ef-b809-ab12ba2601df None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:12.989156 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cbea0-5888-45ef-b809-ab12ba2601df None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:12.990582 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 517/1035] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:12.996732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6e75c-e390-4357-8b21-2d149057d6ec None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:12.997276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6e75c-e390-4357-8b21-2d149057d6ec None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:12.997276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6e75c-e390-4357-8b21-2d149057d6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:13.017667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-5fc6e75c-e390-4357-8b21-2d149057d6ec None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:13.018859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5fc6e75c-e390-4357-8b21-2d149057d6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:13.087463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 519/1036] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:13.237559 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805fe0f-7fa0-4683-bce9-c95604c78b4b None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:13.238115 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805fe0f-7fa0-4683-bce9-c95604c78b4b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:13.238115 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805fe0f-7fa0-4683-bce9-c95604c78b4b None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:13.239616 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 518/1037] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:13.244745 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e074fe9-a462-4d8a-ae02-fa485a268b74 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:13.245035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e074fe9-a462-4d8a-ae02-fa485a268b74 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:13.245306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e074fe9-a462-4d8a-ae02-fa485a268b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:13.265791 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-7e074fe9-a462-4d8a-ae02-fa485a268b74 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:13.267005 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7e074fe9-a462-4d8a-ae02-fa485a268b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:13.340862 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 520/1038] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:13.354124 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805fe0f-7fa0-4683-bce9-c95604c78b4b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:13.459784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805fe0f-7fa0-4683-bce9-c95604c78b4b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:13.493066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:13.499421 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554e253-11da-4249-8bd9-b98f8c948ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:13.500098 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554e253-11da-4249-8bd9-b98f8c948ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:13.500379 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554e253-11da-4249-8bd9-b98f8c948ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:13.500650 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554e253-11da-4249-8bd9-b98f8c948ce1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:13.501812 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554e253-11da-4249-8bd9-b98f8c948ce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:13.503325 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554e253-11da-4249-8bd9-b98f8c948ce1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:13.508066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 519/1039] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:13.511226 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e074fe9-a462-4d8a-ae02-fa485a268b74 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:13.539639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:13.618513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554e253-11da-4249-8bd9-b98f8c948ce1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:13.624881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e074fe9-a462-4d8a-ae02-fa485a268b74 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:13.668371 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386de62c-705e-4f7b-b812-5b7e211b9904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:13.668995 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386de62c-705e-4f7b-b812-5b7e211b9904 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:13.669272 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386de62c-705e-4f7b-b812-5b7e211b9904 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:13.669595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386de62c-705e-4f7b-b812-5b7e211b9904 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:13.671244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386de62c-705e-4f7b-b812-5b7e211b9904 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:13.672990 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386de62c-705e-4f7b-b812-5b7e211b9904 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:13.708026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 521/1040] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:32:13 2022] POST /identity/v3/users => generated 285 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:32:13.742641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554e253-11da-4249-8bd9-b98f8c948ce1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:13.781304 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdc3d1-e8e2-47a3-8847-67a2a7b765e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:13.781949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdc3d1-e8e2-47a3-8847-67a2a7b765e3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:13.782230 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdc3d1-e8e2-47a3-8847-67a2a7b765e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:13.782673 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdc3d1-e8e2-47a3-8847-67a2a7b765e3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:13.783142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfdc3d1-e8e2-47a3-8847-67a2a7b765e3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:13.784723 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfdc3d1-e8e2-47a3-8847-67a2a7b765e3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:13.794527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-3cfdc3d1-e8e2-47a3-8847-67a2a7b765e3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:32:13.794527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:13.794527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:32:13.794527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:32:13.794527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:32:13.794527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:32:13.794527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:32:13.794527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:32:13.794527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:32:13.794527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:13.794527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:32:13.794527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:13.794527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:13.794527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:13.794527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:13.794527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:13.794527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:13.800477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:13.800477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:13.800477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:13.800477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:13.800477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:13.800477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:13.800477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:32:13.800477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:32:13.800477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:32:13.800477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:32:13.800477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:13.800477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:13.800477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:13.800477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:13.800477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:13.800477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:13.801392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:13.801392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:13.801392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:13.801392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:13.801392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:13.801392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:13.801392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:13.801392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:13.801392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:13.801392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:13.801392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:32:13.801392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:32:13.801392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:32:13.801392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:13.801392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 520/1041] 10.222.0.25 () {56 vars in 1105 bytes} [Tue Aug 30 14:32:13 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:13.802229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386de62c-705e-4f7b-b812-5b7e211b9904 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:13.893873 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:13.918524 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386de62c-705e-4f7b-b812-5b7e211b9904 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:13.958172 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c24c5b-b013-47cc-af08-131c7de0d237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:13.959163 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c24c5b-b013-47cc-af08-131c7de0d237 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:13.959659 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c24c5b-b013-47cc-af08-131c7de0d237 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:13.960186 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c24c5b-b013-47cc-af08-131c7de0d237 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:13.961020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c24c5b-b013-47cc-af08-131c7de0d237 None admin] RBAC: Adding query filter params (name=image) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:13.961528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c24c5b-b013-47cc-af08-131c7de0d237 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:13.963085 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c24c5b-b013-47cc-af08-131c7de0d237 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:13.968920 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 522/1042] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:13.974719 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdc3d1-e8e2-47a3-8847-67a2a7b765e3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:14.041537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:14.082768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdc3d1-e8e2-47a3-8847-67a2a7b765e3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:14.117261 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf966eb-da39-4249-adcc-1809791a5803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:14.117878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf966eb-da39-4249-adcc-1809791a5803 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:14.118156 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf966eb-da39-4249-adcc-1809791a5803 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:14.118425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf966eb-da39-4249-adcc-1809791a5803 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:14.119062 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf966eb-da39-4249-adcc-1809791a5803 None admin] RBAC: Adding query filter params (type=image) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:14.119365 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf966eb-da39-4249-adcc-1809791a5803 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:14.120772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf966eb-da39-4249-adcc-1809791a5803 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:14.126067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 521/1043] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:14.132243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c24c5b-b013-47cc-af08-131c7de0d237 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:14.242024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c24c5b-b013-47cc-af08-131c7de0d237 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:14.279696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd261b6-f460-408e-8b05-3b166f0c4f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:14.280735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd261b6-f460-408e-8b05-3b166f0c4f1e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:14.281220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd261b6-f460-408e-8b05-3b166f0c4f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:14.281723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd261b6-f460-408e-8b05-3b166f0c4f1e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:14.282660 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd261b6-f460-408e-8b05-3b166f0c4f1e None admin] RBAC: Adding query filter params (interface=public, service_id=5b9392ba872541ca99c773f153324526, region_id=RegionOne) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:14.283170 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd261b6-f460-408e-8b05-3b166f0c4f1e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:14.284813 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd261b6-f460-408e-8b05-3b166f0c4f1e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:14.290729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 523/1044] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Aug 30 14:32:14 2022] GET /identity/v3/endpoints?service_id=5b9392ba872541ca99c773f153324526&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:14.297131 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf966eb-da39-4249-adcc-1809791a5803 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:14.417862 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf966eb-da39-4249-adcc-1809791a5803 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:14.446595 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:14.455701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098c247-4523-4faa-9168-1f1cbfa93f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:14.456712 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c247-4523-4faa-9168-1f1cbfa93f33 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:14.456972 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c247-4523-4faa-9168-1f1cbfa93f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:14.457630 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c247-4523-4faa-9168-1f1cbfa93f33 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:14.458236 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098c247-4523-4faa-9168-1f1cbfa93f33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:14.459946 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098c247-4523-4faa-9168-1f1cbfa93f33 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:14.467523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 522/1045] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Aug 30 14:32:14 2022] GET /identity/v3/services => generated 1957 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:32:14.768594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea0e77-bdf3-4881-968b-e3c0243567c9 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:14.768875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea0e77-bdf3-4881-968b-e3c0243567c9 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:14.769153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea0e77-bdf3-4881-968b-e3c0243567c9 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:14.770530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 524/1046] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:14.775973 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd2944-2c09-4391-aa5d-5740f654a00f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:14.776291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd2944-2c09-4391-aa5d-5740f654a00f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:14.776560 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd2944-2c09-4391-aa5d-5740f654a00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:14.795311 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1dbd2944-2c09-4391-aa5d-5740f654a00f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:14.796589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1dbd2944-2c09-4391-aa5d-5740f654a00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:14.871451 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 523/1047] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:14 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:15.138446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea0e77-bdf3-4881-968b-e3c0243567c9 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:15.152383 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:15.264694 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea0e77-bdf3-4881-968b-e3c0243567c9 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:15.303410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77ab56-d078-4994-a03a-beb1867c8400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:15.304107 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77ab56-d078-4994-a03a-beb1867c8400 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:15.304378 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77ab56-d078-4994-a03a-beb1867c8400 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:15.304641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77ab56-d078-4994-a03a-beb1867c8400 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:15.319677 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77ab56-d078-4994-a03a-beb1867c8400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:32:15.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-cb77ab56-d078-4994-a03a-beb1867c8400 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:32:15.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:15.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:15.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:15.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:15.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:15.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:15.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:15.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:15.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:15.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:15.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:15.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:32:15.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:32:15.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:32:15.325331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:32:15.326420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:32:15.326420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:32:15.326420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:32:15.326420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:32:15.326420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:15.326420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:15.326420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:15.326420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:15.326420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:15.326420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:15.326420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:15.326420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:15.326420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:15.326420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:15.326420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:15.326420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:15.328629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:15.328629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:15.328629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:15.328629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:15.328629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:32:15.328629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:32:15.328629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:32:15.328629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:32:15.328629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:32:15.328629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:32:15.328629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:32:15.328629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:32:15.328629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 525/1048] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Aug 30 14:32:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:15.333749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd2944-2c09-4391-aa5d-5740f654a00f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:15.445933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd2944-2c09-4391-aa5d-5740f654a00f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:15.472637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:15.486530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49f185-1e05-4080-9082-d7ee84e6c050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:15.487428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49f185-1e05-4080-9082-d7ee84e6c050 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:15.487703 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49f185-1e05-4080-9082-d7ee84e6c050 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:15.487966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49f185-1e05-4080-9082-d7ee84e6c050 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:15.488672 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49f185-1e05-4080-9082-d7ee84e6c050 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:15.488970 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49f185-1e05-4080-9082-d7ee84e6c050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:15.490400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49f185-1e05-4080-9082-d7ee84e6c050 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:15.504499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 524/1049] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Aug 30 14:32:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:15.506380 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77ab56-d078-4994-a03a-beb1867c8400 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:15.584100 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:15.625548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77ab56-d078-4994-a03a-beb1867c8400 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:15.663204 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b877b3-dca2-4c51-b5d3-c593783e9a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:15.663859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b877b3-dca2-4c51-b5d3-c593783e9a9f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:15.664157 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b877b3-dca2-4c51-b5d3-c593783e9a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:15.664423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b877b3-dca2-4c51-b5d3-c593783e9a9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:15.675724 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b877b3-dca2-4c51-b5d3-c593783e9a9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:15.677225 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b877b3-dca2-4c51-b5d3-c593783e9a9f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:15.692338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-13b877b3-dca2-4c51-b5d3-c593783e9a9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:15.692338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:15.692338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:15.692338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:15.692338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:15.692338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:15.692338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:15.692338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:15.692338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:15.692338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:15.692338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:15.692338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:15.692338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:15.692338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:15.692338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:15.692338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:15.693342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:15.693342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:15.693342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:15.693342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:15.693342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:15.693342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:15.693342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:15.693342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:15.693342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:15.693342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:15.693342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:15.693342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:15.693342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:15.693342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:15.693342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:15.693342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:15.693342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:32:15.694239 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:15.694239 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:15.694239 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:32:15.694971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 526/1050] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:15.699528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49f185-1e05-4080-9082-d7ee84e6c050 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:15.717731 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b42651-f709-43d2-97a9-e19fc19d46fe None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:15.718047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b42651-f709-43d2-97a9-e19fc19d46fe None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:15.718272 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b42651-f709-43d2-97a9-e19fc19d46fe None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:15.719752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 527/1051] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:32:15.724591 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b4db9-35d4-4b40-a88d-722f3411a170 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:15.724872 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b4db9-35d4-4b40-a88d-722f3411a170 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:15.725143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b4db9-35d4-4b40-a88d-722f3411a170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:15.743195 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-724b4db9-35d4-4b40-a88d-722f3411a170 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:15.744412 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-724b4db9-35d4-4b40-a88d-722f3411a170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:15.814285 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49f185-1e05-4080-9082-d7ee84e6c050 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:15.821027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 528/1052] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:32:15.852988 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff8b45-d1be-49f5-8aa4-2fe259ea8b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:15.853605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff8b45-d1be-49f5-8aa4-2fe259ea8b01 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:15.853874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff8b45-d1be-49f5-8aa4-2fe259ea8b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:15.854137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff8b45-d1be-49f5-8aa4-2fe259ea8b01 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:15.854778 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff8b45-d1be-49f5-8aa4-2fe259ea8b01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:15.855070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff8b45-d1be-49f5-8aa4-2fe259ea8b01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:15.856480 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff8b45-d1be-49f5-8aa4-2fe259ea8b01 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:15.869908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 525/1053] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:15.875593 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b4db9-35d4-4b40-a88d-722f3411a170 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:15.931618 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:15.982452 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b4db9-35d4-4b40-a88d-722f3411a170 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:16.019783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cb003-7716-407c-8cce-eeaf4f658e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:16.020452 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cb003-7716-407c-8cce-eeaf4f658e86 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:16.020728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cb003-7716-407c-8cce-eeaf4f658e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:16.020990 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cb003-7716-407c-8cce-eeaf4f658e86 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:16.028786 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459cb003-7716-407c-8cce-eeaf4f658e86 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:16.030821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459cb003-7716-407c-8cce-eeaf4f658e86 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:16.043680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-459cb003-7716-407c-8cce-eeaf4f658e86 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:32:16.043680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:16.043680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:16.043680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:16.043680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:16.043680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:16.043680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:16.043680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:16.043680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:16.043680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:16.043680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:16.043680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:16.043680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:32:16.043680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:32:16.043680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:32:16.043680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:16.045402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:16.046392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:16.046392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:16.046392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:32:16.046392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:32:16.046392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:32:16.046392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:32:16.046392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:32:16.046392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:32:16.046392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:32:16.046392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:32:16.048464 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 529/1054] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Aug 30 14:32:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:16.051024 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff8b45-d1be-49f5-8aa4-2fe259ea8b01 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:16.089041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cb003-7716-407c-8cce-eeaf4f658e86 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:16.121008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:16.188889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff8b45-d1be-49f5-8aa4-2fe259ea8b01 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:16.213895 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cb003-7716-407c-8cce-eeaf4f658e86 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:16.226507 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a7953-0479-424f-af45-1d954e03e3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:16.226726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:16.227134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a7953-0479-424f-af45-1d954e03e3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:16.227412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a7953-0479-424f-af45-1d954e03e3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:16.227683 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a7953-0479-424f-af45-1d954e03e3f2 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:16.228460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a7953-0479-424f-af45-1d954e03e3f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:16.228770 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a7953-0479-424f-af45-1d954e03e3f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:16.230210 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a7953-0479-424f-af45-1d954e03e3f2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:16.251079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 526/1055] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Aug 30 14:32:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:16.255637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3470929-b6a2-4696-9e4b-e44f8929eb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:16.256324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3470929-b6a2-4696-9e4b-e44f8929eb89 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:16.256496 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a7953-0479-424f-af45-1d954e03e3f2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:16.256593 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3470929-b6a2-4696-9e4b-e44f8929eb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:16.256855 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3470929-b6a2-4696-9e4b-e44f8929eb89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:16.268685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3470929-b6a2-4696-9e4b-e44f8929eb89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:32:16.273503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-f3470929-b6a2-4696-9e4b-e44f8929eb89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:32:16.273503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:16.273503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:16.273503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:16.273503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:16.273503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:16.273503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:16.273503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:16.273503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:16.273503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:16.273503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:16.273503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:16.273503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:32:16.273503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:32:16.273503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:32:16.273503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:32:16.274624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:32:16.274624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:32:16.274624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:32:16.274624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:32:16.274624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:16.274624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:16.274624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:16.274624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:16.274624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:16.274624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:16.274624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:16.274624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:16.274624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:16.274624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:16.274624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:16.274624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:16.276981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:16.276981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:16.276981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:16.276981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:16.276981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:32:16.276981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:32:16.276981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:32:16.276981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:32:16.276981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:32:16.276981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:32:16.276981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:32:16.276981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:32:16.276981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 530/1056] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:32:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:16.280529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3470929-b6a2-4696-9e4b-e44f8929eb89 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:16.377850 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a7953-0479-424f-af45-1d954e03e3f2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:16.379739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:16.394271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3470929-b6a2-4696-9e4b-e44f8929eb89 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:16.426794 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3aef6-430d-44a5-98bd-9b6e970d6ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:16.427448 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3aef6-430d-44a5-98bd-9b6e970d6ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:16.427724 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3aef6-430d-44a5-98bd-9b6e970d6ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:16.427990 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3aef6-430d-44a5-98bd-9b6e970d6ee8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:16.430764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949974d0-3100-43ec-905b-29a5048e8bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:16.431435 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949974d0-3100-43ec-905b-29a5048e8bcb None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:16.431713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949974d0-3100-43ec-905b-29a5048e8bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:16.431974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949974d0-3100-43ec-905b-29a5048e8bcb None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:16.432694 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949974d0-3100-43ec-905b-29a5048e8bcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:16.432990 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949974d0-3100-43ec-905b-29a5048e8bcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:16.434495 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949974d0-3100-43ec-905b-29a5048e8bcb None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:16.441768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3aef6-430d-44a5-98bd-9b6e970d6ee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:16.443697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3aef6-430d-44a5-98bd-9b6e970d6ee8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:16.446513 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 531/1057] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:32:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:16.452616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949974d0-3100-43ec-905b-29a5048e8bcb None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:16.461744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-fad3aef6-430d-44a5-98bd-9b6e970d6ee8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:16.461744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:16.461744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:16.461744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:16.461744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:16.461744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:16.461744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:16.461744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:16.461744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:16.461744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:16.461744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:16.461744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:16.461744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:16.461744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:16.461744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:16.461744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:16.462739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:16.462739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:16.462739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:16.462739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:16.462739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:16.462739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:16.462739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:16.462739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:16.462739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:16.462739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:16.462739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:16.462739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:16.462739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:16.462739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:16.462739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:16.462739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:16.462739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:32:16.463649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:16.463649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:16.463649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:16.463649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 527/1058] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:32:16.470157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3aef6-430d-44a5-98bd-9b6e970d6ee8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:16.487007 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:16.565072 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949974d0-3100-43ec-905b-29a5048e8bcb None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:16.577236 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3aef6-430d-44a5-98bd-9b6e970d6ee8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:16.601401 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bb5ae-5e2f-48a0-b835-a20b14747130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:16.602044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bb5ae-5e2f-48a0-b835-a20b14747130 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:16.602313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bb5ae-5e2f-48a0-b835-a20b14747130 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:16.602598 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bb5ae-5e2f-48a0-b835-a20b14747130 None admin] PATH_INFO: `/v3/users/b30778f88fdf4c87b023f0a5b4c17ec9` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:16.612241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13706596-a6e9-4a62-9179-62aae6a011f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:16.612926 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13706596-a6e9-4a62-9179-62aae6a011f2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:16.613125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13706596-a6e9-4a62-9179-62aae6a011f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:16.613395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13706596-a6e9-4a62-9179-62aae6a011f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:16.614279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13706596-a6e9-4a62-9179-62aae6a011f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:16.614393 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13706596-a6e9-4a62-9179-62aae6a011f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:16.615730 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13706596-a6e9-4a62-9179-62aae6a011f2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:16.616413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bb5ae-5e2f-48a0-b835-a20b14747130 None admin] RBAC: Authorizing `identity:get_user(user_id=b30778f88fdf4c87b023f0a5b4c17ec9)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:16.618194 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bb5ae-5e2f-48a0-b835-a20b14747130 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:16.619533 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 532/1059] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:32:16 2022] GET /identity/v3/users/b30778f88fdf4c87b023f0a5b4c17ec9 => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:16.620891 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d17249-0480-48fa-bf4c-80f51fd61598 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:16.621169 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d17249-0480-48fa-bf4c-80f51fd61598 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:16.621436 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d17249-0480-48fa-bf4c-80f51fd61598 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:16.622859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 533/1060] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:16.627651 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d17249-0480-48fa-bf4c-80f51fd61598 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:16.633713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 528/1061] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:32:16.635002 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f370bba-86cc-4425-8eb0-4540ea11568a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:16.635280 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f370bba-86cc-4425-8eb0-4540ea11568a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:16.635548 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f370bba-86cc-4425-8eb0-4540ea11568a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:16.654409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9f370bba-86cc-4425-8eb0-4540ea11568a None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:16.655625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9f370bba-86cc-4425-8eb0-4540ea11568a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:16.665379 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:16.734035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:16.738351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 529/1062] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:16.739229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f370bba-86cc-4425-8eb0-4540ea11568a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:16.749608 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d17249-0480-48fa-bf4c-80f51fd61598 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:16.786897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96f3aa-4dcf-4fc7-8ded-47b51409049c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:16.787590 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96f3aa-4dcf-4fc7-8ded-47b51409049c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:16.787772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96f3aa-4dcf-4fc7-8ded-47b51409049c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:16.788063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96f3aa-4dcf-4fc7-8ded-47b51409049c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:16.848642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f370bba-86cc-4425-8eb0-4540ea11568a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:16.864829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96f3aa-4dcf-4fc7-8ded-47b51409049c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:16.866279 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96f3aa-4dcf-4fc7-8ded-47b51409049c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:16.884245 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f127bdf-54f8-4185-9c94-8d9b6d0a41f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:16.884877 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f127bdf-54f8-4185-9c94-8d9b6d0a41f4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:16.885157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f127bdf-54f8-4185-9c94-8d9b6d0a41f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:16.885417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f127bdf-54f8-4185-9c94-8d9b6d0a41f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:16.892443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f127bdf-54f8-4185-9c94-8d9b6d0a41f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:16.893898 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f127bdf-54f8-4185-9c94-8d9b6d0a41f4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:16.905450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-2f127bdf-54f8-4185-9c94-8d9b6d0a41f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:32:16.905450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:16.905450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:16.905450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:16.905450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:16.905450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:16.905450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:16.905450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:16.905450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:16.905450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:16.905450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:16.905450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:16.905450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:32:16.905450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:32:16.905450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:32:16.906491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:32:16.906491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:16.906491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:16.906491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:16.906491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:16.906491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:16.906491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:16.906491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:16.906491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:16.906491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:16.906491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:16.906491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:16.906491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:16.906491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:16.906491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:16.906491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:16.906491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:16.907383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:32:16.907383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:32:16.907383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:32:16.907383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:32:16.907383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:32:16.907383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:32:16.907383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:32:16.907383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:16.907383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 530/1063] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:32:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:16.912964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f127bdf-54f8-4185-9c94-8d9b6d0a41f4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:16.918193 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:16.970111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:16.982132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 534/1064] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:32:16 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:32:16.987781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96f3aa-4dcf-4fc7-8ded-47b51409049c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:17.023498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f127bdf-54f8-4185-9c94-8d9b6d0a41f4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:17.057979 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9ad99-0b4d-48ed-a72b-4db93bf31070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:17.058596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9ad99-0b4d-48ed-a72b-4db93bf31070 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:17.058873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9ad99-0b4d-48ed-a72b-4db93bf31070 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:17.059138 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9ad99-0b4d-48ed-a72b-4db93bf31070 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:17.059835 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9ad99-0b4d-48ed-a72b-4db93bf31070 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:17.061005 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9ad99-0b4d-48ed-a72b-4db93bf31070 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:17.062457 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9ad99-0b4d-48ed-a72b-4db93bf31070 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:17.074249 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 531/1065] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:32:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:17.075302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9ad99-0b4d-48ed-a72b-4db93bf31070 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:17.089599 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96f3aa-4dcf-4fc7-8ded-47b51409049c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:17.126308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa4179-2583-4483-a524-ebf2011107c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:17.126935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa4179-2583-4483-a524-ebf2011107c0 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:17.127213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa4179-2583-4483-a524-ebf2011107c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:17.127489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa4179-2583-4483-a524-ebf2011107c0 None admin] PATH_INFO: `/v3/projects/13c4a60b4a92433eaca615619a729716` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:17.128603 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa4179-2583-4483-a524-ebf2011107c0 None admin] RBAC: Authorizing `identity:get_project(project_id=13c4a60b4a92433eaca615619a729716)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:17.130026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa4179-2583-4483-a524-ebf2011107c0 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:17.131669 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 535/1066] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:32:16 2022] GET /identity/v3/projects/13c4a60b4a92433eaca615619a729716 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:32:17.132425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa4179-2583-4483-a524-ebf2011107c0 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:17.142145 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:17.182291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9ad99-0b4d-48ed-a72b-4db93bf31070 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:17.191020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:17.218606 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea658c72-9f62-4872-8060-8e5a6836f6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:17.219250 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea658c72-9f62-4872-8060-8e5a6836f6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:17.219522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea658c72-9f62-4872-8060-8e5a6836f6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:17.219784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea658c72-9f62-4872-8060-8e5a6836f6f9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:17.220411 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea658c72-9f62-4872-8060-8e5a6836f6f9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:17.221790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea658c72-9f62-4872-8060-8e5a6836f6f9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:17.231281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-ea658c72-9f62-4872-8060-8e5a6836f6f9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:32:17.231281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:17.231281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:32:17.231281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:32:17.231281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:32:17.231281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:32:17.231281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:32:17.231281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:32:17.231281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:32:17.231281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:17.231281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:32:17.231281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:17.231281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:17.231281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:17.231281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:17.231281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:17.231281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:17.232366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:17.232366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:17.232366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:17.232366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:17.232366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:17.232366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:17.232366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:32:17.232366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:32:17.232366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:32:17.232366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:32:17.232366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:17.232366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:17.232366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:17.232366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:17.232366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:17.232366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:17.233238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:17.233238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:17.233238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:17.233238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:17.233238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:17.233238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:17.233238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:17.233238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:17.233238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:17.233238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:17.233238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:32:17.233238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:32:17.233238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:32:17.233238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:17.233238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 532/1067] 10.222.0.25 () {56 vars in 1105 bytes} [Tue Aug 30 14:32:17 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:17.234073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea658c72-9f62-4872-8060-8e5a6836f6f9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:17.240383 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa4179-2583-4483-a524-ebf2011107c0 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:17.275213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e61d0-15df-4db8-bf21-a4562dcf0882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:17.275882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e61d0-15df-4db8-bf21-a4562dcf0882 None admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:17.276197 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e61d0-15df-4db8-bf21-a4562dcf0882 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:17.276459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e61d0-15df-4db8-bf21-a4562dcf0882 None admin] PATH_INFO: `/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/cc95bbdafb0c4f34ad0662bc2e3ba4ec/roles/2295f599ff36442faecde157de2c08c8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:17.278621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e61d0-15df-4db8-bf21-a4562dcf0882 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9c57a581324217ad2e64a0c9c7fdee, user_id=cc95bbdafb0c4f34ad0662bc2e3ba4ec, role_id=2295f599ff36442faecde157de2c08c8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:17.280233 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e61d0-15df-4db8-bf21-a4562dcf0882 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:17.295480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 536/1068] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:32:17 2022] PUT /identity/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/cc95bbdafb0c4f34ad0662bc2e3ba4ec/roles/2295f599ff36442faecde157de2c08c8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:32:17.296363 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e61d0-15df-4db8-bf21-a4562dcf0882 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:17.368869 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea658c72-9f62-4872-8060-8e5a6836f6f9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:17.407146 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:17.417906 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902edf52-8652-4025-a59c-470e98ab62ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:17.418556 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902edf52-8652-4025-a59c-470e98ab62ea None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:17.418837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902edf52-8652-4025-a59c-470e98ab62ea None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:17.419109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902edf52-8652-4025-a59c-470e98ab62ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:17.420703 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902edf52-8652-4025-a59c-470e98ab62ea None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=13c4a60b4a92433eaca615619a729716, user.id=b30778f88fdf4c87b023f0a5b4c17ec9) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:17.421020 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902edf52-8652-4025-a59c-470e98ab62ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:17.422527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902edf52-8652-4025-a59c-470e98ab62ea None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:17.428844 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 533/1069] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:32:17 2022] GET /identity/v3/role_assignments?user.id=b30778f88fdf4c87b023f0a5b4c17ec9&scope.project.id=13c4a60b4a92433eaca615619a729716&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:17.430479 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e61d0-15df-4db8-bf21-a4562dcf0882 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:17.462874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:17.484398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa333c4-faf7-40af-a718-37dc7ec08436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:17.485316 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa333c4-faf7-40af-a718-37dc7ec08436 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:17.485778 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa333c4-faf7-40af-a718-37dc7ec08436 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:17.486228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa333c4-faf7-40af-a718-37dc7ec08436 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:17.487061 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa333c4-faf7-40af-a718-37dc7ec08436 None admin] RBAC: Adding query filter params (name=image) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:17.487518 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa333c4-faf7-40af-a718-37dc7ec08436 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:17.489132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa333c4-faf7-40af-a718-37dc7ec08436 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:17.495618 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 537/1070] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:17.501618 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902edf52-8652-4025-a59c-470e98ab62ea None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:17.617747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902edf52-8652-4025-a59c-470e98ab62ea None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:17.655661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6d96d-a813-416d-a1f8-dc42cfdf4a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:17.656315 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6d96d-a813-416d-a1f8-dc42cfdf4a0d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:17.656588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6d96d-a813-416d-a1f8-dc42cfdf4a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:17.656848 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6d96d-a813-416d-a1f8-dc42cfdf4a0d None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:17.657536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6d96d-a813-416d-a1f8-dc42cfdf4a0d None admin] RBAC: Adding query filter params (type=image) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:17.657777 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6d96d-a813-416d-a1f8-dc42cfdf4a0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:17.659135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6d96d-a813-416d-a1f8-dc42cfdf4a0d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:17.664614 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 534/1071] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:17.670657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa333c4-faf7-40af-a718-37dc7ec08436 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:17.789060 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa333c4-faf7-40af-a718-37dc7ec08436 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:17.826815 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74161b-1813-43b5-9e1f-fe4e541095a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:17.827428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74161b-1813-43b5-9e1f-fe4e541095a7 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:17.827707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74161b-1813-43b5-9e1f-fe4e541095a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:17.827977 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74161b-1813-43b5-9e1f-fe4e541095a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:17.828685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df74161b-1813-43b5-9e1f-fe4e541095a7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:17.830058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df74161b-1813-43b5-9e1f-fe4e541095a7 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:17.856848 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 538/1072] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:32:17 2022] POST /identity/v3/endpoints => generated 327 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:32:19.220438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3ad8e-f35e-4fca-948c-b8c5b2e9c9dd None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:19.220994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3ad8e-f35e-4fca-948c-b8c5b2e9c9dd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:19.220994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3ad8e-f35e-4fca-948c-b8c5b2e9c9dd None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:19.222272 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 535/1073] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:19.227802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764d4d4-2804-415b-9885-13aa09cb9f19 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:19.228274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764d4d4-2804-415b-9885-13aa09cb9f19 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:19.228391 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764d4d4-2804-415b-9885-13aa09cb9f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:19.247153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-0764d4d4-2804-415b-9885-13aa09cb9f19 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:19.248690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0764d4d4-2804-415b-9885-13aa09cb9f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:19.279934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:19.336144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 539/1074] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:19.353046 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0163454-4541-4935-ac8a-583e98ac09e6 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:19.353371 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0163454-4541-4935-ac8a-583e98ac09e6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:19.353601 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0163454-4541-4935-ac8a-583e98ac09e6 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:19.354934 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 536/1075] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:19.361214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d1f6f-edd8-454d-ad6e-0887722a7bf6 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:19.361485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d1f6f-edd8-454d-ad6e-0887722a7bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:19.361779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d1f6f-edd8-454d-ad6e-0887722a7bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:19.381414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-d96d1f6f-edd8-454d-ad6e-0887722a7bf6 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:19.382624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d96d1f6f-edd8-454d-ad6e-0887722a7bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:19.461764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 540/1076] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:19.595227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0163454-4541-4935-ac8a-583e98ac09e6 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:19.662935 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:19.711181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0163454-4541-4935-ac8a-583e98ac09e6 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:19.734771 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d1f6f-edd8-454d-ad6e-0887722a7bf6 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:19.748257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f9eb7-112d-4ef7-9e30-7f2b5b81c816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:19.748904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f9eb7-112d-4ef7-9e30-7f2b5b81c816 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:19.749183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f9eb7-112d-4ef7-9e30-7f2b5b81c816 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:19.749448 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f9eb7-112d-4ef7-9e30-7f2b5b81c816 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:19.761475 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f9eb7-112d-4ef7-9e30-7f2b5b81c816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:32:19.766359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-fa3f9eb7-112d-4ef7-9e30-7f2b5b81c816 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:32:19.766359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:19.766359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:19.766359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:19.766359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:19.766359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:19.766359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:19.766359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:19.766359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:19.766359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:19.766359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:19.766359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:19.766359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:32:19.766359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:32:19.766359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:32:19.766359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:32:19.767434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:32:19.767434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:32:19.767434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:32:19.767434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:32:19.767434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:19.767434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:19.767434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:19.767434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:19.767434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:19.767434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:19.767434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:19.767434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:19.767434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:19.767434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:19.767434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:19.767434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:19.768687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:19.768687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:19.768687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:19.768687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:19.768687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:32:19.768687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:32:19.768687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:32:19.768687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:32:19.768687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:32:19.768687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:32:19.768687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:32:19.768687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:19.768687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 537/1077] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Aug 30 14:32:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:19.774356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d51ae2-50e5-4783-9bc0-30629e6a800b None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:19.774798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d51ae2-50e5-4783-9bc0-30629e6a800b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:19.775231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d51ae2-50e5-4783-9bc0-30629e6a800b None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:19.777327 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 538/1078] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:32:19.778456 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d51ae2-50e5-4783-9bc0-30629e6a800b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:19.870698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:19.878665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d1f6f-edd8-454d-ad6e-0887722a7bf6 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:19.911848 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d51ae2-50e5-4783-9bc0-30629e6a800b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:19.917685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006fc4b-b709-4c45-a3ac-08543d18bfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:19.918409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006fc4b-b709-4c45-a3ac-08543d18bfa6 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:19.918709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006fc4b-b709-4c45-a3ac-08543d18bfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:19.918968 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006fc4b-b709-4c45-a3ac-08543d18bfa6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:19.930738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:19.933907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5006fc4b-b709-4c45-a3ac-08543d18bfa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:32:19.938781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-5006fc4b-b709-4c45-a3ac-08543d18bfa6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:32:19.938781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:19.938781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:19.938781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:19.938781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:19.938781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:19.938781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:19.938781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:19.938781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:19.938781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:19.938781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:19.938781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:19.938781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:32:19.938781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:32:19.938781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:32:19.938781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:32:19.939732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:32:19.939732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:32:19.939732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:32:19.939732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:32:19.939732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:19.939732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:19.939732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:19.939732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:19.939732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:19.939732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:19.939732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:19.939732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:19.939732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:19.939732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:19.939732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:19.939732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:19.940650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:19.940650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:19.940650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:19.940650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:19.940650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:32:19.940650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:32:19.940650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:32:19.940650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:32:19.940650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:32:19.940650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:32:19.940650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:32:19.940650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:32:19.940650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 541/1079] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:32:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:32:19.941354 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b161a2e-696f-4520-bcf1-5bc97b0ff55f None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:19.942056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b161a2e-696f-4520-bcf1-5bc97b0ff55f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:19.942056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b161a2e-696f-4520-bcf1-5bc97b0ff55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:19.954078 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b6efc-a957-4e7a-b59d-e13de2105d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:19.954706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b6efc-a957-4e7a-b59d-e13de2105d53 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:19.954973 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b6efc-a957-4e7a-b59d-e13de2105d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:19.955238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b6efc-a957-4e7a-b59d-e13de2105d53 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:19.956286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b6efc-a957-4e7a-b59d-e13de2105d53 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:19.956580 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b6efc-a957-4e7a-b59d-e13de2105d53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:19.957943 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b6efc-a957-4e7a-b59d-e13de2105d53 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:19.960448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-5b161a2e-696f-4520-bcf1-5bc97b0ff55f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:19.961647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5b161a2e-696f-4520-bcf1-5bc97b0ff55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:19.966765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 539/1080] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Aug 30 14:32:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:19.967593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b6efc-a957-4e7a-b59d-e13de2105d53 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:20.033507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 542/1081] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:20.034411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b161a2e-696f-4520-bcf1-5bc97b0ff55f None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:20.080024 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b6efc-a957-4e7a-b59d-e13de2105d53 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:20.133368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f862f8-fe75-47cf-96ee-113456883b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:20.134232 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f862f8-fe75-47cf-96ee-113456883b29 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:20.134505 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f862f8-fe75-47cf-96ee-113456883b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:20.134773 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f862f8-fe75-47cf-96ee-113456883b29 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:20.135458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f862f8-fe75-47cf-96ee-113456883b29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:20.135756 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f862f8-fe75-47cf-96ee-113456883b29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:20.137183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f862f8-fe75-47cf-96ee-113456883b29 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:20.145602 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 540/1082] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:32:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:20.146067 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:20.151482 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f862f8-fe75-47cf-96ee-113456883b29 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:20.164623 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b161a2e-696f-4520-bcf1-5bc97b0ff55f None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:20.198821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:20.203798 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e4a57-7495-4e7f-a675-409a9e7694dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:20.204860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e4a57-7495-4e7f-a675-409a9e7694dd None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:20.205421 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e4a57-7495-4e7f-a675-409a9e7694dd None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:20.205696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e4a57-7495-4e7f-a675-409a9e7694dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:20.220345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e4a57-7495-4e7f-a675-409a9e7694dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:20.221906 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e4a57-7495-4e7f-a675-409a9e7694dd None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:20.237863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-a56e4a57-7495-4e7f-a675-409a9e7694dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:20.237863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:20.237863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:20.237863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:20.237863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:20.237863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:20.237863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:20.237863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:20.237863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:20.237863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:20.237863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:20.237863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:20.237863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:20.237863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:20.237863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:20.237863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:20.239725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:20.239725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:20.239725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:20.239725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:20.239725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:20.239725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:20.239725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:20.239725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:20.239725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:20.239725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:20.239725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:20.239725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:20.239725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:20.239725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:20.239725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:20.239725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:20.239725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:32:20.240874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:20.240874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:20.240874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:32:20.240874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 543/1083] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:20.244921 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e4a57-7495-4e7f-a675-409a9e7694dd None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:20.265296 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f862f8-fe75-47cf-96ee-113456883b29 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:20.302491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e883732-be8f-406c-84f1-b6bb451ee3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:20.303244 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e883732-be8f-406c-84f1-b6bb451ee3dc None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:20.303521 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e883732-be8f-406c-84f1-b6bb451ee3dc None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:20.303790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e883732-be8f-406c-84f1-b6bb451ee3dc None admin] PATH_INFO: `/v3/users/b30778f88fdf4c87b023f0a5b4c17ec9` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:20.304937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e883732-be8f-406c-84f1-b6bb451ee3dc None admin] RBAC: Authorizing `identity:get_user(user_id=b30778f88fdf4c87b023f0a5b4c17ec9)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:20.306375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e883732-be8f-406c-84f1-b6bb451ee3dc None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:20.308146 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 541/1084] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:32:20 2022] GET /identity/v3/users/b30778f88fdf4c87b023f0a5b4c17ec9 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:20.308759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e883732-be8f-406c-84f1-b6bb451ee3dc None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:20.355390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e4a57-7495-4e7f-a675-409a9e7694dd None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:20.390428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed93f29-dd1b-4648-b682-3a81092b3695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:20.390959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed93f29-dd1b-4648-b682-3a81092b3695 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:20.391227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed93f29-dd1b-4648-b682-3a81092b3695 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:20.391488 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed93f29-dd1b-4648-b682-3a81092b3695 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:20.392162 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed93f29-dd1b-4648-b682-3a81092b3695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:20.392463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed93f29-dd1b-4648-b682-3a81092b3695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:20.394111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed93f29-dd1b-4648-b682-3a81092b3695 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:20.397123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:20.408389 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 544/1085] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:32:20.409106 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e883732-be8f-406c-84f1-b6bb451ee3dc None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:20.409443 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed93f29-dd1b-4648-b682-3a81092b3695 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:20.421870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:20.447641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c6391-4896-437d-a179-5f3a27c954ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:20.448289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c6391-4896-437d-a179-5f3a27c954ae None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:20.448569 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c6391-4896-437d-a179-5f3a27c954ae None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:20.448840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c6391-4896-437d-a179-5f3a27c954ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:20.524503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed93f29-dd1b-4648-b682-3a81092b3695 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:20.527258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c6391-4896-437d-a179-5f3a27c954ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:20.528837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c6391-4896-437d-a179-5f3a27c954ae None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:20.569913 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acf4c9-493b-48e8-af83-4aa57d7bd582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:20.570554 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acf4c9-493b-48e8-af83-4aa57d7bd582 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:20.570827 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acf4c9-493b-48e8-af83-4aa57d7bd582 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:20.571072 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acf4c9-493b-48e8-af83-4aa57d7bd582 None admin] PATH_INFO: `/v3/projects/13c4a60b4a92433eaca615619a729716` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:20.572331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7acf4c9-493b-48e8-af83-4aa57d7bd582 None admin] RBAC: Authorizing `identity:get_project(project_id=13c4a60b4a92433eaca615619a729716)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:20.573781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7acf4c9-493b-48e8-af83-4aa57d7bd582 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:20.575435 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 545/1086] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:32:20 2022] GET /identity/v3/projects/13c4a60b4a92433eaca615619a729716 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:20.576484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acf4c9-493b-48e8-af83-4aa57d7bd582 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:20.640914 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 542/1087] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:32:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:32:20.641684 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c6391-4896-437d-a179-5f3a27c954ae None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:20.648576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:20.652424 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:20.689828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acf4c9-493b-48e8-af83-4aa57d7bd582 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:20.728109 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2c105-bdfe-4052-bb9a-546e19795230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:20.728740 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2c105-bdfe-4052-bb9a-546e19795230 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:20.729012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2c105-bdfe-4052-bb9a-546e19795230 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:20.729272 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2c105-bdfe-4052-bb9a-546e19795230 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:20.737593 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2c105-bdfe-4052-bb9a-546e19795230 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:20.739026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2c105-bdfe-4052-bb9a-546e19795230 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:20.750434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c6391-4896-437d-a179-5f3a27c954ae None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:20.751541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-d7a2c105-bdfe-4052-bb9a-546e19795230 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:32:20.751541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:20.751541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:20.751541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:20.751541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:20.751541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:20.751541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:20.751541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:20.751541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:20.751541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:20.751541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:20.751541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:20.751541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:32:20.751541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:32:20.751541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:32:20.751541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:20.752705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:20.753678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:20.753678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:20.753678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:32:20.753678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:32:20.753678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:32:20.753678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:32:20.753678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:32:20.753678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:32:20.753678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:32:20.753678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:32:20.753678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 546/1088] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Aug 30 14:32:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:20.753678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2c105-bdfe-4052-bb9a-546e19795230 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:20.793567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1282a0-2410-4d6c-a61f-5af785b69e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:20.794255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1282a0-2410-4d6c-a61f-5af785b69e91 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:20.794706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1282a0-2410-4d6c-a61f-5af785b69e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:20.794977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1282a0-2410-4d6c-a61f-5af785b69e91 None admin] PATH_INFO: `/v3/projects/13c4a60b4a92433eaca615619a729716/users/b30778f88fdf4c87b023f0a5b4c17ec9/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:20.797090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1282a0-2410-4d6c-a61f-5af785b69e91 None admin] RBAC: Authorizing `identity:create_grant(project_id=13c4a60b4a92433eaca615619a729716, user_id=b30778f88fdf4c87b023f0a5b4c17ec9, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:20.798529 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1282a0-2410-4d6c-a61f-5af785b69e91 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:20.813421 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 543/1089] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:32:20 2022] PUT /identity/v3/projects/13c4a60b4a92433eaca615619a729716/users/b30778f88fdf4c87b023f0a5b4c17ec9/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:32:20.814386 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1282a0-2410-4d6c-a61f-5af785b69e91 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:20.883481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2c105-bdfe-4052-bb9a-546e19795230 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:20.901346 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:20.911847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:20.930556 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071ee62-06a4-475d-bd7a-acadad5ea7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:20.931220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071ee62-06a4-475d-bd7a-acadad5ea7dc None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:20.931690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071ee62-06a4-475d-bd7a-acadad5ea7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:20.931959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071ee62-06a4-475d-bd7a-acadad5ea7dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:20.948240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071ee62-06a4-475d-bd7a-acadad5ea7dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:20.949732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071ee62-06a4-475d-bd7a-acadad5ea7dc None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:20.957267 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1282a0-2410-4d6c-a61f-5af785b69e91 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:20.970150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-4071ee62-06a4-475d-bd7a-acadad5ea7dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:20.970150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:20.970150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:20.970150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:20.970150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:20.970150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:20.970150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:20.970150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:20.970150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:20.970150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:20.970150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:20.970150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:20.970150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:20.970150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:20.970150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:20.970150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:20.971180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:20.971180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:20.971180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:20.971180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:20.971180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:20.971180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:20.971180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:20.971180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:20.971180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:20.971180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:20.971180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:20.971180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:20.971180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:20.971180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:20.971180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:20.971180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:20.971180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:32:20.972113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:20.972113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:20.972113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:32:20.972113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 547/1090] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:20.977879 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071ee62-06a4-475d-bd7a-acadad5ea7dc None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:21.001838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd652a-809a-4462-a127-75bcc33dfe77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:21.002515 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd652a-809a-4462-a127-75bcc33dfe77 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:21.002736 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd652a-809a-4462-a127-75bcc33dfe77 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:21.003003 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd652a-809a-4462-a127-75bcc33dfe77 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:21.003747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd652a-809a-4462-a127-75bcc33dfe77 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:21.004080 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd652a-809a-4462-a127-75bcc33dfe77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:21.005516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd652a-809a-4462-a127-75bcc33dfe77 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:21.024038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 544/1091] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Aug 30 14:32:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:21.030410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd652a-809a-4462-a127-75bcc33dfe77 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:21.113663 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071ee62-06a4-475d-bd7a-acadad5ea7dc None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:21.152958 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705c5d7-ae6b-47ea-8f5d-ec693b8ff99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:21.153494 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705c5d7-ae6b-47ea-8f5d-ec693b8ff99e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:21.153928 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705c5d7-ae6b-47ea-8f5d-ec693b8ff99e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:21.154871 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705c5d7-ae6b-47ea-8f5d-ec693b8ff99e None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:21.155544 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705c5d7-ae6b-47ea-8f5d-ec693b8ff99e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:21.156034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705c5d7-ae6b-47ea-8f5d-ec693b8ff99e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:21.157370 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705c5d7-ae6b-47ea-8f5d-ec693b8ff99e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:21.158532 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd652a-809a-4462-a127-75bcc33dfe77 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:21.171031 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 548/1092] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:32:21.192965 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:21.205634 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f2d87-446f-4af6-8a4a-3f284a52cc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:21.205861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f2d87-446f-4af6-8a4a-3f284a52cc48 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:21.206572 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f2d87-446f-4af6-8a4a-3f284a52cc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:21.206869 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f2d87-446f-4af6-8a4a-3f284a52cc48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:21.294989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f2d87-446f-4af6-8a4a-3f284a52cc48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:21.296595 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f2d87-446f-4af6-8a4a-3f284a52cc48 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:21.428221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 545/1093] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:32:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 14:32:21.434268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705c5d7-ae6b-47ea-8f5d-ec693b8ff99e None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:21.477830 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:21.544576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705c5d7-ae6b-47ea-8f5d-ec693b8ff99e None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:21.578120 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5e18a-b571-43a3-b729-265c6f11f380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:21.578800 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5e18a-b571-43a3-b729-265c6f11f380 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:21.579110 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5e18a-b571-43a3-b729-265c6f11f380 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:21.579412 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5e18a-b571-43a3-b729-265c6f11f380 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:21.591080 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5e18a-b571-43a3-b729-265c6f11f380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:21.592933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5e18a-b571-43a3-b729-265c6f11f380 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:21.609835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-a2d5e18a-b571-43a3-b729-265c6f11f380 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:21.609835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:21.609835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:21.609835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:21.609835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:21.609835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:21.609835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:21.609835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:21.609835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:21.609835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:21.609835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:21.609835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:21.609835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:21.609835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:21.609835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:21.609835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:21.611026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:21.611026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:21.611026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:21.611026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:21.611026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:21.611026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:21.611026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:21.611026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:21.611026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:21.611026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:21.611026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:21.611026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:21.611026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:21.611026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:21.611026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:21.611026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:21.611026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:32:21.611919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:21.611919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:21.611919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:32:21.611919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 549/1094] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:21.617228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f2d87-446f-4af6-8a4a-3f284a52cc48 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:21.648537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:21.732327 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f2d87-446f-4af6-8a4a-3f284a52cc48 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:21.767149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5215398-1758-4692-8e3f-bd07ec7bf523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:21.767754 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5215398-1758-4692-8e3f-bd07ec7bf523 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:21.768052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5215398-1758-4692-8e3f-bd07ec7bf523 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:21.768322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5215398-1758-4692-8e3f-bd07ec7bf523 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:21.768966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5215398-1758-4692-8e3f-bd07ec7bf523 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:21.769264 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5215398-1758-4692-8e3f-bd07ec7bf523 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:21.770635 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5215398-1758-4692-8e3f-bd07ec7bf523 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:21.782497 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 546/1095] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:21.788366 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5e18a-b571-43a3-b729-265c6f11f380 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:21.883512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:21.897766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5e18a-b571-43a3-b729-265c6f11f380 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:21.936333 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2df3c-b7a4-4b43-9e54-4d931f3516a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:21.936970 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2df3c-b7a4-4b43-9e54-4d931f3516a5 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:21.937244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2df3c-b7a4-4b43-9e54-4d931f3516a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:21.937496 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2df3c-b7a4-4b43-9e54-4d931f3516a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:21.944355 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2df3c-b7a4-4b43-9e54-4d931f3516a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:21.945782 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2df3c-b7a4-4b43-9e54-4d931f3516a5 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:21.957664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-fca2df3c-b7a4-4b43-9e54-4d931f3516a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:32:21.957664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:21.957664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:21.957664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:21.957664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:21.957664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:21.957664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:21.957664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:21.957664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:21.957664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:21.957664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:21.957664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:21.957664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:32:21.957664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:32:21.957664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:32:21.958616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:32:21.958616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:21.958616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:21.958616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:21.958616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:21.958616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:21.958616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:21.958616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:21.958616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:21.958616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:21.958616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:21.958616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:21.958616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:21.958616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:21.958616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:21.958616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:21.958616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:21.959546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:32:21.959546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:32:21.959546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:32:21.959546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:32:21.959546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:32:21.959546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:32:21.959546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:32:21.959546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:32:21.959546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 550/1096] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:32:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:21.964886 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5215398-1758-4692-8e3f-bd07ec7bf523 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:22.054194 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:22.071037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5215398-1758-4692-8e3f-bd07ec7bf523 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:22.107646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bdb4e-2590-4710-99e7-961fa0b7fdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:22.108291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bdb4e-2590-4710-99e7-961fa0b7fdd6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:22.108570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bdb4e-2590-4710-99e7-961fa0b7fdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:22.108844 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bdb4e-2590-4710-99e7-961fa0b7fdd6 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:22.109541 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bdb4e-2590-4710-99e7-961fa0b7fdd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:22.109844 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bdb4e-2590-4710-99e7-961fa0b7fdd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:22.111264 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bdb4e-2590-4710-99e7-961fa0b7fdd6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:22.124248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 547/1097] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:32:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:22.130423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2df3c-b7a4-4b43-9e54-4d931f3516a5 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:22.235540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2df3c-b7a4-4b43-9e54-4d931f3516a5 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:22.269276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5965d6-6aa3-42cb-b319-8f529484030d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:22.269883 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5965d6-6aa3-42cb-b319-8f529484030d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:22.270158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5965d6-6aa3-42cb-b319-8f529484030d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:22.270421 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5965d6-6aa3-42cb-b319-8f529484030d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:22.271143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5965d6-6aa3-42cb-b319-8f529484030d None admin] RBAC: Adding query filter params (role.id=2295f599ff36442faecde157de2c08c8, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=cc95bbdafb0c4f34ad0662bc2e3ba4ec) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:22.271442 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5965d6-6aa3-42cb-b319-8f529484030d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:22.272904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5965d6-6aa3-42cb-b319-8f529484030d None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:22.278593 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 551/1098] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:32:22 2022] GET /identity/v3/role_assignments?user.id=cc95bbdafb0c4f34ad0662bc2e3ba4ec&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=2295f599ff36442faecde157de2c08c8 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:22.741803 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adbf1a-56a0-4017-ba04-f97523fbc520 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:22.742377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adbf1a-56a0-4017-ba04-f97523fbc520 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:22.742377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adbf1a-56a0-4017-ba04-f97523fbc520 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:22.743646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 548/1099] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:22.749488 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b8c2f-0c90-4c29-a14f-150e5bfa81ea None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:22.749958 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b8c2f-0c90-4c29-a14f-150e5bfa81ea None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:22.750044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b8c2f-0c90-4c29-a14f-150e5bfa81ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:22.769320 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-1b8b8c2f-0c90-4c29-a14f-150e5bfa81ea None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:22.770525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-1b8b8c2f-0c90-4c29-a14f-150e5bfa81ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:22.778507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:22.842011 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 552/1100] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:23.105309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adbf1a-56a0-4017-ba04-f97523fbc520 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:23.214119 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adbf1a-56a0-4017-ba04-f97523fbc520 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:23.249390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff272a83-b462-46a6-8070-6189903a20c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:23.250017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff272a83-b462-46a6-8070-6189903a20c0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:23.250288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff272a83-b462-46a6-8070-6189903a20c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:23.250551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff272a83-b462-46a6-8070-6189903a20c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:23.261109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff272a83-b462-46a6-8070-6189903a20c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:32:23.266028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-ff272a83-b462-46a6-8070-6189903a20c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:32:23.266028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:23.266028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:23.266028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:23.266028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:23.266028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:23.266028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:23.266028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:23.266028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:23.266028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:23.266028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:23.266028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:23.266028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:32:23.266028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:32:23.266028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:32:23.266028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:32:23.267069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:32:23.267069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:32:23.267069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:32:23.267069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:32:23.267069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:23.267069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:23.267069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:23.267069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:23.267069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:23.267069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:23.267069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:23.267069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:23.267069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:23.267069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:23.267069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:23.267069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:23.268053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:23.268053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:23.268053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:23.268053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:23.268053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:32:23.268053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:32:23.268053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:32:23.268053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:32:23.268053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:32:23.268053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:32:23.268053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:32:23.268053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:23.268053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 549/1101] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:32:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:23.273492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b8c2f-0c90-4c29-a14f-150e5bfa81ea None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:23.379085 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b8c2f-0c90-4c29-a14f-150e5bfa81ea None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:23.414746 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca007bf-c82c-42a9-a7df-dd81f80338e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:23.415382 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca007bf-c82c-42a9-a7df-dd81f80338e0 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:23.415658 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca007bf-c82c-42a9-a7df-dd81f80338e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:23.415929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca007bf-c82c-42a9-a7df-dd81f80338e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:23.416621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca007bf-c82c-42a9-a7df-dd81f80338e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:23.416921 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca007bf-c82c-42a9-a7df-dd81f80338e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:23.418302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca007bf-c82c-42a9-a7df-dd81f80338e0 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:23.427226 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 553/1102] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:32:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:23.433355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff272a83-b462-46a6-8070-6189903a20c0 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:23.443352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:23.540161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff272a83-b462-46a6-8070-6189903a20c0 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:23.577377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd9a4d-bfae-4b38-960e-62fe23349feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:23.578021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd9a4d-bfae-4b38-960e-62fe23349feb None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:23.578291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd9a4d-bfae-4b38-960e-62fe23349feb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:23.578552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd9a4d-bfae-4b38-960e-62fe23349feb None admin] PATH_INFO: `/v3/users/b30778f88fdf4c87b023f0a5b4c17ec9` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:23.580061 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd9a4d-bfae-4b38-960e-62fe23349feb None admin] RBAC: Authorizing `identity:get_user(user_id=b30778f88fdf4c87b023f0a5b4c17ec9)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:23.582679 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd9a4d-bfae-4b38-960e-62fe23349feb None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:23.584948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 550/1103] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:32:23 2022] GET /identity/v3/users/b30778f88fdf4c87b023f0a5b4c17ec9 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:23.593952 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca007bf-c82c-42a9-a7df-dd81f80338e0 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:23.599750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:23.703936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca007bf-c82c-42a9-a7df-dd81f80338e0 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:23.739440 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e2fc9-971a-40a2-ad73-135e373650df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:23.740028 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e2fc9-971a-40a2-ad73-135e373650df None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:23.740351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e2fc9-971a-40a2-ad73-135e373650df None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:23.740620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e2fc9-971a-40a2-ad73-135e373650df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:23.809633 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631e2fc9-971a-40a2-ad73-135e373650df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:23.811106 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631e2fc9-971a-40a2-ad73-135e373650df None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:23.813751 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:23.914543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 554/1104] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:32:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:32:23.921173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd9a4d-bfae-4b38-960e-62fe23349feb None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:24.002475 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:24.030724 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd9a4d-bfae-4b38-960e-62fe23349feb None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:24.068748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdde56-f142-45a2-8948-0a8286e0223e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:24.069384 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdde56-f142-45a2-8948-0a8286e0223e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:24.069672 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdde56-f142-45a2-8948-0a8286e0223e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:24.069930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdde56-f142-45a2-8948-0a8286e0223e None admin] PATH_INFO: `/v3/projects/13c4a60b4a92433eaca615619a729716` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:24.071056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdde56-f142-45a2-8948-0a8286e0223e None admin] RBAC: Authorizing `identity:get_project(project_id=13c4a60b4a92433eaca615619a729716)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:24.072530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdde56-f142-45a2-8948-0a8286e0223e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:24.074157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 551/1105] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:32:23 2022] GET /identity/v3/projects/13c4a60b4a92433eaca615619a729716 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:24.081345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e2fc9-971a-40a2-ad73-135e373650df None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:24.205020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e2fc9-971a-40a2-ad73-135e373650df None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:24.207829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:24.242267 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fde2b4-3e5a-40fd-9d23-42167e67502c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:24.242864 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fde2b4-3e5a-40fd-9d23-42167e67502c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:24.243141 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fde2b4-3e5a-40fd-9d23-42167e67502c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:24.243422 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fde2b4-3e5a-40fd-9d23-42167e67502c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:24.244170 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fde2b4-3e5a-40fd-9d23-42167e67502c None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=13c4a60b4a92433eaca615619a729716, user.id=b30778f88fdf4c87b023f0a5b4c17ec9) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:24.244469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fde2b4-3e5a-40fd-9d23-42167e67502c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:24.245887 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fde2b4-3e5a-40fd-9d23-42167e67502c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:24.251936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 555/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:32:24 2022] GET /identity/v3/role_assignments?user.id=b30778f88fdf4c87b023f0a5b4c17ec9&scope.project.id=13c4a60b4a92433eaca615619a729716&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:26.204497 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89780f-3e0c-4f5f-81cb-28aa7023f14b None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:26.205126 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89780f-3e0c-4f5f-81cb-28aa7023f14b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:26.205126 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89780f-3e0c-4f5f-81cb-28aa7023f14b None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:26.206762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 552/1107] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:26.212228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8adfe-8cbf-44e5-a674-27d01c82cc23 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:26.212678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8adfe-8cbf-44e5-a674-27d01c82cc23 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:26.212773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8adfe-8cbf-44e5-a674-27d01c82cc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:26.232335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-bfe8adfe-8cbf-44e5-a674-27d01c82cc23 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:26.233559 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-bfe8adfe-8cbf-44e5-a674-27d01c82cc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:26.304924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 556/1108] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:26.595318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89780f-3e0c-4f5f-81cb-28aa7023f14b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:26.707345 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89780f-3e0c-4f5f-81cb-28aa7023f14b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:26.742346 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc4168-1af9-43e5-a7a1-fc03b206fef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:26.742980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc4168-1af9-43e5-a7a1-fc03b206fef7 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:26.743253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc4168-1af9-43e5-a7a1-fc03b206fef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:26.743518 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc4168-1af9-43e5-a7a1-fc03b206fef7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:26.744646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc4168-1af9-43e5-a7a1-fc03b206fef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:26.746088 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc4168-1af9-43e5-a7a1-fc03b206fef7 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:26.747716 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 553/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:26.753732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8adfe-8cbf-44e5-a674-27d01c82cc23 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:26.855224 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:26.861300 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8adfe-8cbf-44e5-a674-27d01c82cc23 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:26.902275 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31e6bf-d11b-4963-9428-173e46691f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:26.902880 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31e6bf-d11b-4963-9428-173e46691f73 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:26.903154 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31e6bf-d11b-4963-9428-173e46691f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:26.903422 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31e6bf-d11b-4963-9428-173e46691f73 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:26.904439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31e6bf-d11b-4963-9428-173e46691f73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:26.906274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31e6bf-d11b-4963-9428-173e46691f73 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:26.929278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 557/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:32:26 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:32:28.863040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2ce40-ecb3-4835-8651-ad3bcd701499 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:28.863632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2ce40-ecb3-4835-8651-ad3bcd701499 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:28.863632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2ce40-ecb3-4835-8651-ad3bcd701499 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:28.864957 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 554/1111] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:28.870314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63d62e-e79b-435c-abf6-e6ad73596493 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:28.870759 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63d62e-e79b-435c-abf6-e6ad73596493 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:28.870850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63d62e-e79b-435c-abf6-e6ad73596493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:28.889893 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-7f63d62e-e79b-435c-abf6-e6ad73596493 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:28.890990 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7f63d62e-e79b-435c-abf6-e6ad73596493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:28.960491 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 558/1112] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:29.237307 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2ce40-ecb3-4835-8651-ad3bcd701499 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:29.250328 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:29.342449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2ce40-ecb3-4835-8651-ad3bcd701499 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:29.377015 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce3bff-2ba4-4f6d-a906-aecc391bc40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:29.377634 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce3bff-2ba4-4f6d-a906-aecc391bc40c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:29.377905 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce3bff-2ba4-4f6d-a906-aecc391bc40c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:29.378165 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce3bff-2ba4-4f6d-a906-aecc391bc40c None admin] PATH_INFO: `/v3/roles/aedf4e1e5c7848099d7bd183778f5896` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:29.379581 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce3bff-2ba4-4f6d-a906-aecc391bc40c None admin] RBAC: Authorizing `identity:get_role(role_id=aedf4e1e5c7848099d7bd183778f5896)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:29.380966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce3bff-2ba4-4f6d-a906-aecc391bc40c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:29.382589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 555/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:32:29 2022] GET /identity/v3/roles/aedf4e1e5c7848099d7bd183778f5896 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:29.388718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63d62e-e79b-435c-abf6-e6ad73596493 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:29.474790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:29.492181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63d62e-e79b-435c-abf6-e6ad73596493 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:29.527914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbb8d4-388d-435c-9414-72fbeb1576cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:29.528593 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbb8d4-388d-435c-9414-72fbeb1576cd None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:29.528873 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbb8d4-388d-435c-9414-72fbeb1576cd None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:29.529135 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbb8d4-388d-435c-9414-72fbeb1576cd None admin] PATH_INFO: `/v3/users/914db877a9504136940f6e4d8ff2ec26` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:29.542928 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbb8d4-388d-435c-9414-72fbeb1576cd None admin] RBAC: Authorizing `identity:get_user(user_id=914db877a9504136940f6e4d8ff2ec26)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:29.544487 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbb8d4-388d-435c-9414-72fbeb1576cd None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:29.546300 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 559/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:32:29 2022] GET /identity/v3/users/914db877a9504136940f6e4d8ff2ec26 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:29.552471 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce3bff-2ba4-4f6d-a906-aecc391bc40c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:29.633671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:29.655688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce3bff-2ba4-4f6d-a906-aecc391bc40c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:29.691071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8de8df-afd8-4382-8416-de3065a061c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:29.691673 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8de8df-afd8-4382-8416-de3065a061c3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:29.691944 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8de8df-afd8-4382-8416-de3065a061c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:29.692262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8de8df-afd8-4382-8416-de3065a061c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:29.763437 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8de8df-afd8-4382-8416-de3065a061c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:29.764924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8de8df-afd8-4382-8416-de3065a061c3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:29.843044 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:29.867178 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 556/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:32:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:32:29.873732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbb8d4-388d-435c-9414-72fbeb1576cd None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:29.978206 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbb8d4-388d-435c-9414-72fbeb1576cd None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:30.017291 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662592fd-30a9-4207-b692-94af1314c047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:30.017885 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662592fd-30a9-4207-b692-94af1314c047 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:30.018164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662592fd-30a9-4207-b692-94af1314c047 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:30.018432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662592fd-30a9-4207-b692-94af1314c047 None admin] PATH_INFO: `/v3/projects/13c4a60b4a92433eaca615619a729716` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:30.019794 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662592fd-30a9-4207-b692-94af1314c047 None admin] RBAC: Authorizing `identity:get_project(project_id=13c4a60b4a92433eaca615619a729716)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:30.021287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662592fd-30a9-4207-b692-94af1314c047 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:30.022958 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 560/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:32:29 2022] GET /identity/v3/projects/13c4a60b4a92433eaca615619a729716 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:30.029577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8de8df-afd8-4382-8416-de3065a061c3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:30.135869 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8de8df-afd8-4382-8416-de3065a061c3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:30.190023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde1a81-f11c-41b7-b47b-a60066b7180e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:30.191028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde1a81-f11c-41b7-b47b-a60066b7180e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:30.191514 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde1a81-f11c-41b7-b47b-a60066b7180e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:30.191983 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde1a81-f11c-41b7-b47b-a60066b7180e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:30.193330 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde1a81-f11c-41b7-b47b-a60066b7180e None admin] RBAC: Adding query filter params (role.id=aedf4e1e5c7848099d7bd183778f5896, scope.project.id=13c4a60b4a92433eaca615619a729716, user.id=914db877a9504136940f6e4d8ff2ec26) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:30.193859 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde1a81-f11c-41b7-b47b-a60066b7180e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:30.196554 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde1a81-f11c-41b7-b47b-a60066b7180e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:30.205603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 557/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:32:30 2022] GET /identity/v3/role_assignments?user.id=914db877a9504136940f6e4d8ff2ec26&scope.project.id=13c4a60b4a92433eaca615619a729716&role.id=aedf4e1e5c7848099d7bd183778f5896 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:32.074445 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f18d4-eea5-4630-bea4-fe820476806e None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:32.074998 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f18d4-eea5-4630-bea4-fe820476806e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:32.074998 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f18d4-eea5-4630-bea4-fe820476806e None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:32.076509 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 561/1118] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:32.083431 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2089d6a-f89f-4392-93fe-8ff83873b8c3 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:32.083917 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2089d6a-f89f-4392-93fe-8ff83873b8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:32.084054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2089d6a-f89f-4392-93fe-8ff83873b8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:32.104850 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e2089d6a-f89f-4392-93fe-8ff83873b8c3 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:32.106066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e2089d6a-f89f-4392-93fe-8ff83873b8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:32.144486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:32.181332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 558/1119] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:32.470989 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f18d4-eea5-4630-bea4-fe820476806e None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:32.484612 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:32.587655 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f18d4-eea5-4630-bea4-fe820476806e None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:32.624312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36877d6f-bfdf-46e4-b497-bbf57c9954d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:32.624945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36877d6f-bfdf-46e4-b497-bbf57c9954d1 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:32.625199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36877d6f-bfdf-46e4-b497-bbf57c9954d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:32.625489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36877d6f-bfdf-46e4-b497-bbf57c9954d1 None admin] PATH_INFO: `/v3/roles/aedf4e1e5c7848099d7bd183778f5896` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:32.626944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36877d6f-bfdf-46e4-b497-bbf57c9954d1 None admin] RBAC: Authorizing `identity:get_role(role_id=aedf4e1e5c7848099d7bd183778f5896)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:32.628345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36877d6f-bfdf-46e4-b497-bbf57c9954d1 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:32.629702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 562/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:32:32 2022] GET /identity/v3/roles/aedf4e1e5c7848099d7bd183778f5896 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:32.635801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2089d6a-f89f-4392-93fe-8ff83873b8c3 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:32.742411 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2089d6a-f89f-4392-93fe-8ff83873b8c3 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:32.779514 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f317e18-8559-44ed-ab8f-a7260d8378f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:32.780189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f317e18-8559-44ed-ab8f-a7260d8378f9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:32.780464 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f317e18-8559-44ed-ab8f-a7260d8378f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:32.780727 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f317e18-8559-44ed-ab8f-a7260d8378f9 None admin] PATH_INFO: `/v3/users/914db877a9504136940f6e4d8ff2ec26` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:32.782001 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f317e18-8559-44ed-ab8f-a7260d8378f9 None admin] RBAC: Authorizing `identity:get_user(user_id=914db877a9504136940f6e4d8ff2ec26)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:32.783451 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f317e18-8559-44ed-ab8f-a7260d8378f9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:32.785078 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 559/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:32:32 2022] GET /identity/v3/users/914db877a9504136940f6e4d8ff2ec26 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:32.791268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36877d6f-bfdf-46e4-b497-bbf57c9954d1 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:32.875625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:32.898965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36877d6f-bfdf-46e4-b497-bbf57c9954d1 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:32.935318 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1195de-4eea-4ea4-ae78-421cf93c034a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:32.935953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1195de-4eea-4ea4-ae78-421cf93c034a None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:32.936265 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1195de-4eea-4ea4-ae78-421cf93c034a None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:32.936564 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1195de-4eea-4ea4-ae78-421cf93c034a None admin] PATH_INFO: `/v3/projects/13c4a60b4a92433eaca615619a729716` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:32.937754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1195de-4eea-4ea4-ae78-421cf93c034a None admin] RBAC: Authorizing `identity:get_project(project_id=13c4a60b4a92433eaca615619a729716)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:32.939185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1195de-4eea-4ea4-ae78-421cf93c034a None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:32.940878 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 563/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:32:32 2022] GET /identity/v3/projects/13c4a60b4a92433eaca615619a729716 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:32.947202 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f317e18-8559-44ed-ab8f-a7260d8378f9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:32.991353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:33.053352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f317e18-8559-44ed-ab8f-a7260d8378f9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:33.089068 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2b8e3-89dc-455c-aa1b-7756bb03b1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:33.089766 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2b8e3-89dc-455c-aa1b-7756bb03b1bc None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:33.089985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2b8e3-89dc-455c-aa1b-7756bb03b1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:33.090252 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2b8e3-89dc-455c-aa1b-7756bb03b1bc None admin] PATH_INFO: `/v3/projects/13c4a60b4a92433eaca615619a729716/users/914db877a9504136940f6e4d8ff2ec26/roles/aedf4e1e5c7848099d7bd183778f5896` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:33.092709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f2b8e3-89dc-455c-aa1b-7756bb03b1bc None admin] RBAC: Authorizing `identity:create_grant(project_id=13c4a60b4a92433eaca615619a729716, user_id=914db877a9504136940f6e4d8ff2ec26, role_id=aedf4e1e5c7848099d7bd183778f5896)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:33.094142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f2b8e3-89dc-455c-aa1b-7756bb03b1bc None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:33.109318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 560/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:32:32 2022] PUT /identity/v3/projects/13c4a60b4a92433eaca615619a729716/users/914db877a9504136940f6e4d8ff2ec26/roles/aedf4e1e5c7848099d7bd183778f5896 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:32:34.976130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462566f-f95a-4246-b103-27f91c0a27e2 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:34.976684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462566f-f95a-4246-b103-27f91c0a27e2 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:34.976684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462566f-f95a-4246-b103-27f91c0a27e2 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:34.977926 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 564/1124] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:34.983704 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab5e2f-2a76-441a-a5b3-c7b0bb4e49b0 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:34.984186 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab5e2f-2a76-441a-a5b3-c7b0bb4e49b0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:34.984296 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab5e2f-2a76-441a-a5b3-c7b0bb4e49b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:35.003789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-60ab5e2f-2a76-441a-a5b3-c7b0bb4e49b0 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:35.005043 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-60ab5e2f-2a76-441a-a5b3-c7b0bb4e49b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:35.077229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 561/1125] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:35.339917 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462566f-f95a-4246-b103-27f91c0a27e2 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:35.451197 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462566f-f95a-4246-b103-27f91c0a27e2 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:35.486971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea8994-7fc3-4f0d-853f-7291247cf5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:35.487601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea8994-7fc3-4f0d-853f-7291247cf5d6 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:35.487868 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea8994-7fc3-4f0d-853f-7291247cf5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:35.488145 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea8994-7fc3-4f0d-853f-7291247cf5d6 None admin] PATH_INFO: `/v3/roles/aedf4e1e5c7848099d7bd183778f5896` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:35.489541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea8994-7fc3-4f0d-853f-7291247cf5d6 None admin] RBAC: Authorizing `identity:get_role(role_id=aedf4e1e5c7848099d7bd183778f5896)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:35.490904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea8994-7fc3-4f0d-853f-7291247cf5d6 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:35.492285 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 565/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:32:35 2022] GET /identity/v3/roles/aedf4e1e5c7848099d7bd183778f5896 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:35.498951 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab5e2f-2a76-441a-a5b3-c7b0bb4e49b0 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:35.534811 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:35.621981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab5e2f-2a76-441a-a5b3-c7b0bb4e49b0 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:35.656547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff8779-22da-4997-a0ee-a8983943e08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:35.657186 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff8779-22da-4997-a0ee-a8983943e08c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:35.657492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff8779-22da-4997-a0ee-a8983943e08c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:35.657754 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff8779-22da-4997-a0ee-a8983943e08c None admin] PATH_INFO: `/v3/users/914db877a9504136940f6e4d8ff2ec26` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:35.658935 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff8779-22da-4997-a0ee-a8983943e08c None admin] RBAC: Authorizing `identity:get_user(user_id=914db877a9504136940f6e4d8ff2ec26)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:35.660412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff8779-22da-4997-a0ee-a8983943e08c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:35.661999 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 562/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:32:35 2022] GET /identity/v3/users/914db877a9504136940f6e4d8ff2ec26 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:35.668167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea8994-7fc3-4f0d-853f-7291247cf5d6 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:35.685230 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:35.772625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea8994-7fc3-4f0d-853f-7291247cf5d6 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:35.807816 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7a6c5-9354-4132-8d5a-fc1052434ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:35.808443 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7a6c5-9354-4132-8d5a-fc1052434ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:35.808717 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7a6c5-9354-4132-8d5a-fc1052434ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:35.808987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7a6c5-9354-4132-8d5a-fc1052434ab2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:35.879451 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c7a6c5-9354-4132-8d5a-fc1052434ab2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:35.880846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c7a6c5-9354-4132-8d5a-fc1052434ab2 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:35.896783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:36.007027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 566/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:32:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:32:36.017066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff8779-22da-4997-a0ee-a8983943e08c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:36.120442 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:36.124523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff8779-22da-4997-a0ee-a8983943e08c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:36.160723 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa5ba2-9838-43f8-a452-7d76ea0698fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:36.161277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa5ba2-9838-43f8-a452-7d76ea0698fe None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:36.161524 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa5ba2-9838-43f8-a452-7d76ea0698fe None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:36.161818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa5ba2-9838-43f8-a452-7d76ea0698fe None admin] PATH_INFO: `/v3/projects/13c4a60b4a92433eaca615619a729716` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:36.162986 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa5ba2-9838-43f8-a452-7d76ea0698fe None admin] RBAC: Authorizing `identity:get_project(project_id=13c4a60b4a92433eaca615619a729716)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:36.164452 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa5ba2-9838-43f8-a452-7d76ea0698fe None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:36.166103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 563/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:32:36 2022] GET /identity/v3/projects/13c4a60b4a92433eaca615619a729716 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:36.172861 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7a6c5-9354-4132-8d5a-fc1052434ab2 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:36.287001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7a6c5-9354-4132-8d5a-fc1052434ab2 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:36.303493 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:36.327885 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd8c05-26b3-4aff-98e0-3fb7ad5a41ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:36.328548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd8c05-26b3-4aff-98e0-3fb7ad5a41ed None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:36.328825 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd8c05-26b3-4aff-98e0-3fb7ad5a41ed None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:36.329091 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd8c05-26b3-4aff-98e0-3fb7ad5a41ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:36.329825 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd8c05-26b3-4aff-98e0-3fb7ad5a41ed None admin] RBAC: Adding query filter params (role.id=aedf4e1e5c7848099d7bd183778f5896, scope.project.id=13c4a60b4a92433eaca615619a729716, user.id=914db877a9504136940f6e4d8ff2ec26) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:36.330130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd8c05-26b3-4aff-98e0-3fb7ad5a41ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:36.331929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd8c05-26b3-4aff-98e0-3fb7ad5a41ed None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:36.338677 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 567/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:32:36 2022] GET /identity/v3/role_assignments?user.id=914db877a9504136940f6e4d8ff2ec26&scope.project.id=13c4a60b4a92433eaca615619a729716&role.id=aedf4e1e5c7848099d7bd183778f5896 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:38.508965 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688966d7-8db0-4eb8-919c-f2c8b982230d None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:38.509531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688966d7-8db0-4eb8-919c-f2c8b982230d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:38.509531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688966d7-8db0-4eb8-919c-f2c8b982230d None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:38.510960 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 564/1131] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:38.517744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26974671-198b-4afc-908a-278dbe00eb4f None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:38.518213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26974671-198b-4afc-908a-278dbe00eb4f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:38.518556 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26974671-198b-4afc-908a-278dbe00eb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:38.538842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-26974671-198b-4afc-908a-278dbe00eb4f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:38.540043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-26974671-198b-4afc-908a-278dbe00eb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:38.615749 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 568/1132] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:38.923863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688966d7-8db0-4eb8-919c-f2c8b982230d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:39.038441 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688966d7-8db0-4eb8-919c-f2c8b982230d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:39.075598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400da397-2744-4cc1-8d97-9d81f2c3d0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:39.076426 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400da397-2744-4cc1-8d97-9d81f2c3d0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:39.076752 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400da397-2744-4cc1-8d97-9d81f2c3d0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:39.077025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400da397-2744-4cc1-8d97-9d81f2c3d0d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:39.078263 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400da397-2744-4cc1-8d97-9d81f2c3d0d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:39.079726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400da397-2744-4cc1-8d97-9d81f2c3d0d4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:39.081746 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 565/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:39.087989 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26974671-198b-4afc-908a-278dbe00eb4f None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:39.210600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26974671-198b-4afc-908a-278dbe00eb4f None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:39.220656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:39.247679 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83398042-aac6-4852-8b0d-177d5ca080a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:39.248307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83398042-aac6-4852-8b0d-177d5ca080a5 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:39.248593 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83398042-aac6-4852-8b0d-177d5ca080a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:39.248842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83398042-aac6-4852-8b0d-177d5ca080a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:39.249507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83398042-aac6-4852-8b0d-177d5ca080a5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:39.250928 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83398042-aac6-4852-8b0d-177d5ca080a5 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:39.279897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 569/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Aug 30 14:32:39 2022] POST /identity/v3/projects => generated 311 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:32:41.390035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d291db-ffc5-4f26-893e-3b4a797a1df8 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:41.390594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d291db-ffc5-4f26-893e-3b4a797a1df8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:41.390594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d291db-ffc5-4f26-893e-3b4a797a1df8 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:41.391919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 566/1135] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:41.398032 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8620d-a1ce-45ab-8ede-835db9e66c54 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:41.398578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8620d-a1ce-45ab-8ede-835db9e66c54 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:41.398578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8620d-a1ce-45ab-8ede-835db9e66c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:41.419710 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-53c8620d-a1ce-45ab-8ede-835db9e66c54 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:41.420997 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-53c8620d-a1ce-45ab-8ede-835db9e66c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:41.493443 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 570/1136] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:41.775687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d291db-ffc5-4f26-893e-3b4a797a1df8 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:41.831188 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:41.903619 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d291db-ffc5-4f26-893e-3b4a797a1df8 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:41.947907 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6db279-e73d-4e90-a7d3-1467b66d986c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:41.949004 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6db279-e73d-4e90-a7d3-1467b66d986c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:41.949482 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6db279-e73d-4e90-a7d3-1467b66d986c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:41.949940 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6db279-e73d-4e90-a7d3-1467b66d986c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:41.951870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6db279-e73d-4e90-a7d3-1467b66d986c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:41.954516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6db279-e73d-4e90-a7d3-1467b66d986c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:41.957729 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 567/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:41.963980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8620d-a1ce-45ab-8ede-835db9e66c54 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:42.063280 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:42.077785 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8620d-a1ce-45ab-8ede-835db9e66c54 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:42.132729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56526ce0-31bb-46cd-805f-a72ce2537a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:42.133345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56526ce0-31bb-46cd-805f-a72ce2537a8f None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:42.133589 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56526ce0-31bb-46cd-805f-a72ce2537a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:42.133855 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56526ce0-31bb-46cd-805f-a72ce2537a8f None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:42.134533 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56526ce0-31bb-46cd-805f-a72ce2537a8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:42.135963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56526ce0-31bb-46cd-805f-a72ce2537a8f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:42.164626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 571/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:32:41 2022] POST /identity/v3/users => generated 286 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:32:44.145809 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bae8f-662a-48d3-ab39-a1a2e659ed37 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:44.146369 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bae8f-662a-48d3-ab39-a1a2e659ed37 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:44.146369 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bae8f-662a-48d3-ab39-a1a2e659ed37 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:44.147702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 568/1139] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:44.153177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ee8d8-132a-40ac-b702-a84d569bb3c0 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:44.153891 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ee8d8-132a-40ac-b702-a84d569bb3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:44.153891 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ee8d8-132a-40ac-b702-a84d569bb3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:44.173302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-cc9ee8d8-132a-40ac-b702-a84d569bb3c0 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:44.174509 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-cc9ee8d8-132a-40ac-b702-a84d569bb3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:44.246454 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 572/1140] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:44.525204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bae8f-662a-48d3-ab39-a1a2e659ed37 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:44.635101 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bae8f-662a-48d3-ab39-a1a2e659ed37 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:44.636722 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:44.673684 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec21d24-2bc8-446a-88c3-abf162b9b3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:44.674313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec21d24-2bc8-446a-88c3-abf162b9b3ef None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:44.674586 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec21d24-2bc8-446a-88c3-abf162b9b3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:44.674847 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec21d24-2bc8-446a-88c3-abf162b9b3ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:44.685808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec21d24-2bc8-446a-88c3-abf162b9b3ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:32:44.690671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-bec21d24-2bc8-446a-88c3-abf162b9b3ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:32:44.690671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:44.690671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:44.690671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:44.690671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:44.690671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:44.690671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:44.690671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:44.690671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:44.690671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:44.690671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:44.690671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:44.690671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:32:44.690671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:32:44.690671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:32:44.690671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:32:44.691768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:32:44.691768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:32:44.691768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:32:44.691768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:32:44.691768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:44.691768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:44.691768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:44.691768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:44.691768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:44.691768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:44.691768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:44.691768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:44.691768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:44.691768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:44.691768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:44.691768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:44.693040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:44.693040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:44.693040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:44.693040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:44.693040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:32:44.693040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:32:44.693040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:32:44.693040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:32:44.693040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:32:44.693040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:32:44.693040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:32:44.693040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:44.693040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 569/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:32:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:44.698068 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ee8d8-132a-40ac-b702-a84d569bb3c0 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:44.781918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:44.808136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ee8d8-132a-40ac-b702-a84d569bb3c0 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:44.846668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80947bc7-4378-4bb6-b537-ff6f4beb4ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:44.847369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80947bc7-4378-4bb6-b537-ff6f4beb4ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:44.847652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80947bc7-4378-4bb6-b537-ff6f4beb4ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:44.847929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80947bc7-4378-4bb6-b537-ff6f4beb4ca9 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:44.848636 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80947bc7-4378-4bb6-b537-ff6f4beb4ca9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:44.848938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80947bc7-4378-4bb6-b537-ff6f4beb4ca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:44.850308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80947bc7-4378-4bb6-b537-ff6f4beb4ca9 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:44.858883 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 573/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:32:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:44.865731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec21d24-2bc8-446a-88c3-abf162b9b3ef None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:44.976392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec21d24-2bc8-446a-88c3-abf162b9b3ef None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:45.014832 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b904b0-ec1a-49e6-9123-741454f63163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:45.015453 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b904b0-ec1a-49e6-9123-741454f63163 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:45.015726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b904b0-ec1a-49e6-9123-741454f63163 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:45.015989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b904b0-ec1a-49e6-9123-741454f63163 None admin] PATH_INFO: `/v3/users/6f000ecf1679491cbc24f7900c0894ec` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:45.029046 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b904b0-ec1a-49e6-9123-741454f63163 None admin] RBAC: Authorizing `identity:get_user(user_id=6f000ecf1679491cbc24f7900c0894ec)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:45.030490 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b904b0-ec1a-49e6-9123-741454f63163 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:45.032215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 570/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:32:44 2022] GET /identity/v3/users/6f000ecf1679491cbc24f7900c0894ec => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:45.038513 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80947bc7-4378-4bb6-b537-ff6f4beb4ca9 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:45.144145 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80947bc7-4378-4bb6-b537-ff6f4beb4ca9 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:45.180220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46c192-bf76-434d-99f2-08652c1a0790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:45.180846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46c192-bf76-434d-99f2-08652c1a0790 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:45.181086 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46c192-bf76-434d-99f2-08652c1a0790 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:45.181352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46c192-bf76-434d-99f2-08652c1a0790 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:45.190301 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:45.251623 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46c192-bf76-434d-99f2-08652c1a0790 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:45.253105 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46c192-bf76-434d-99f2-08652c1a0790 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:45.358991 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 574/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:32:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:32:45.365094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b904b0-ec1a-49e6-9123-741454f63163 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:45.388410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:45.466701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b904b0-ec1a-49e6-9123-741454f63163 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:45.507065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f43e6-5eee-4b58-91e3-051e9ddf5fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:45.507706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f43e6-5eee-4b58-91e3-051e9ddf5fcf None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:45.507979 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f43e6-5eee-4b58-91e3-051e9ddf5fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:45.508283 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f43e6-5eee-4b58-91e3-051e9ddf5fcf None admin] PATH_INFO: `/v3/projects/23a2477dee1c4148bced7c591aaf6c9b` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:45.509422 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f43e6-5eee-4b58-91e3-051e9ddf5fcf None admin] RBAC: Authorizing `identity:get_project(project_id=23a2477dee1c4148bced7c591aaf6c9b)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:45.510862 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f43e6-5eee-4b58-91e3-051e9ddf5fcf None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:45.512562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 571/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:32:45 2022] GET /identity/v3/projects/23a2477dee1c4148bced7c591aaf6c9b => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:45.519610 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46c192-bf76-434d-99f2-08652c1a0790 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:45.569796 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:45.643933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46c192-bf76-434d-99f2-08652c1a0790 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:45.681125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f26af9-d507-4006-a9d8-1a9c7a66feed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:45.681731 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f26af9-d507-4006-a9d8-1a9c7a66feed None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:45.682005 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f26af9-d507-4006-a9d8-1a9c7a66feed None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:45.682276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f26af9-d507-4006-a9d8-1a9c7a66feed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:45.682998 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f26af9-d507-4006-a9d8-1a9c7a66feed None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=23a2477dee1c4148bced7c591aaf6c9b, user.id=6f000ecf1679491cbc24f7900c0894ec) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:45.683304 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f26af9-d507-4006-a9d8-1a9c7a66feed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:45.684884 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f26af9-d507-4006-a9d8-1a9c7a66feed None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:45.690606 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 575/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:32:45 2022] GET /identity/v3/role_assignments?user.id=6f000ecf1679491cbc24f7900c0894ec&scope.project.id=23a2477dee1c4148bced7c591aaf6c9b&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:47.553937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729fda5-6e17-4a06-b630-5dcb53bf8825 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:47.554503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729fda5-6e17-4a06-b630-5dcb53bf8825 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:47.554503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729fda5-6e17-4a06-b630-5dcb53bf8825 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:47.556209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 572/1147] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:47.561802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd5b6e-c20b-4d82-9bbb-1e013e7534cc None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:47.562321 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd5b6e-c20b-4d82-9bbb-1e013e7534cc None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:47.562321 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd5b6e-c20b-4d82-9bbb-1e013e7534cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:47.582745 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-50cd5b6e-c20b-4d82-9bbb-1e013e7534cc None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:47.583951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-50cd5b6e-c20b-4d82-9bbb-1e013e7534cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:47.658135 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 576/1148] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:47.919462 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729fda5-6e17-4a06-b630-5dcb53bf8825 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:48.021808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:48.038169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729fda5-6e17-4a06-b630-5dcb53bf8825 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:48.073489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81baba-cbf0-4a70-87f1-ecbf9875291a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:48.074088 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81baba-cbf0-4a70-87f1-ecbf9875291a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:48.074358 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81baba-cbf0-4a70-87f1-ecbf9875291a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:48.074623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81baba-cbf0-4a70-87f1-ecbf9875291a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:48.086397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81baba-cbf0-4a70-87f1-ecbf9875291a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:32:48.091629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-cc81baba-cbf0-4a70-87f1-ecbf9875291a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:32:48.091629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:48.091629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:48.091629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:48.091629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:48.091629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:48.091629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:48.091629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:48.091629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:48.091629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:48.091629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:48.091629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:48.091629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:32:48.091629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:32:48.091629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:32:48.091629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:32:48.092741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:32:48.092741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:32:48.092741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:32:48.092741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:32:48.092741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:48.092741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:48.092741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:48.092741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:48.092741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:48.092741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:48.092741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:48.092741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:48.092741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:48.092741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:48.092741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:48.092741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:48.093748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:48.093748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:48.093748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:48.093748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:48.093748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:32:48.093748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:32:48.093748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:32:48.093748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:32:48.093748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:32:48.093748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:32:48.093748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:32:48.093748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:48.093748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 573/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:32:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:48.099264 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd5b6e-c20b-4d82-9bbb-1e013e7534cc None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:48.138869 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:48.215882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd5b6e-c20b-4d82-9bbb-1e013e7534cc None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:48.250993 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1e6a9-27cf-447e-8f30-0a30b35c9245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:48.251601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1e6a9-27cf-447e-8f30-0a30b35c9245 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:48.251873 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1e6a9-27cf-447e-8f30-0a30b35c9245 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:48.252171 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1e6a9-27cf-447e-8f30-0a30b35c9245 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:48.252847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf1e6a9-27cf-447e-8f30-0a30b35c9245 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:48.253140 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf1e6a9-27cf-447e-8f30-0a30b35c9245 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:48.254544 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf1e6a9-27cf-447e-8f30-0a30b35c9245 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:48.264695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 577/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:32:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:48.270757 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81baba-cbf0-4a70-87f1-ecbf9875291a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:48.378677 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81baba-cbf0-4a70-87f1-ecbf9875291a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:48.426919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34278dc6-d2fa-420a-8dba-3ea92433e82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:48.427563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34278dc6-d2fa-420a-8dba-3ea92433e82b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:48.427849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34278dc6-d2fa-420a-8dba-3ea92433e82b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:48.428349 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34278dc6-d2fa-420a-8dba-3ea92433e82b None admin] PATH_INFO: `/v3/users/6f000ecf1679491cbc24f7900c0894ec` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:48.429922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34278dc6-d2fa-420a-8dba-3ea92433e82b None admin] RBAC: Authorizing `identity:get_user(user_id=6f000ecf1679491cbc24f7900c0894ec)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:48.431397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34278dc6-d2fa-420a-8dba-3ea92433e82b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:48.433315 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 574/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:32:48 2022] GET /identity/v3/users/6f000ecf1679491cbc24f7900c0894ec => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:48.439359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1e6a9-27cf-447e-8f30-0a30b35c9245 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:48.550345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:48.558491 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1e6a9-27cf-447e-8f30-0a30b35c9245 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:48.599348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853223cf-8fc2-4e8f-8aa8-45421cfc455f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:48.600273 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853223cf-8fc2-4e8f-8aa8-45421cfc455f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:48.600273 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853223cf-8fc2-4e8f-8aa8-45421cfc455f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:48.600542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853223cf-8fc2-4e8f-8aa8-45421cfc455f None admin] PATH_INFO: `/v3/projects/23a2477dee1c4148bced7c591aaf6c9b` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:48.601715 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853223cf-8fc2-4e8f-8aa8-45421cfc455f None admin] RBAC: Authorizing `identity:get_project(project_id=23a2477dee1c4148bced7c591aaf6c9b)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:48.603152 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853223cf-8fc2-4e8f-8aa8-45421cfc455f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:48.604859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 578/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:32:48 2022] GET /identity/v3/projects/23a2477dee1c4148bced7c591aaf6c9b => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:48.613345 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34278dc6-d2fa-420a-8dba-3ea92433e82b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:48.634576 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:48.737298 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34278dc6-d2fa-420a-8dba-3ea92433e82b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:48.773859 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca769901-bac9-4c36-95ca-bc48487cb155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:48.774523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca769901-bac9-4c36-95ca-bc48487cb155 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:48.774796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca769901-bac9-4c36-95ca-bc48487cb155 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:48.775065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca769901-bac9-4c36-95ca-bc48487cb155 None admin] PATH_INFO: `/v3/projects/23a2477dee1c4148bced7c591aaf6c9b/users/6f000ecf1679491cbc24f7900c0894ec/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:48.776999 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca769901-bac9-4c36-95ca-bc48487cb155 None admin] RBAC: Authorizing `identity:create_grant(project_id=23a2477dee1c4148bced7c591aaf6c9b, user_id=6f000ecf1679491cbc24f7900c0894ec, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:48.778605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca769901-bac9-4c36-95ca-bc48487cb155 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:48.797311 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 575/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:32:48 2022] PUT /identity/v3/projects/23a2477dee1c4148bced7c591aaf6c9b/users/6f000ecf1679491cbc24f7900c0894ec/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:32:50.746482 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670cb53-621d-4ab8-919e-91403f55b614 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:50.747103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670cb53-621d-4ab8-919e-91403f55b614 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:50.747103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670cb53-621d-4ab8-919e-91403f55b614 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:50.748633 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 579/1154] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:50.754068 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdc8d7-d9cf-4fec-909f-56b3039cdf5c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:50.754552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdc8d7-d9cf-4fec-909f-56b3039cdf5c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:50.754647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdc8d7-d9cf-4fec-909f-56b3039cdf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:50.774153 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-76cdc8d7-d9cf-4fec-909f-56b3039cdf5c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:50.775378 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-76cdc8d7-d9cf-4fec-909f-56b3039cdf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:50.849891 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 576/1155] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:51.117369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670cb53-621d-4ab8-919e-91403f55b614 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:51.225900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670cb53-621d-4ab8-919e-91403f55b614 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:51.265781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e27e5-9f33-40d5-878b-c49042a821a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:51.266414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e27e5-9f33-40d5-878b-c49042a821a9 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:51.266688 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e27e5-9f33-40d5-878b-c49042a821a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:51.266952 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e27e5-9f33-40d5-878b-c49042a821a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:51.278479 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e27e5-9f33-40d5-878b-c49042a821a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:32:51.283210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-2c8e27e5-9f33-40d5-878b-c49042a821a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:32:51.283210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:51.283210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:51.283210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:51.283210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:51.283210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:51.283210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:51.283210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:51.283210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:51.283210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:51.283210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:51.283210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:51.283210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:32:51.283210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:32:51.283210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:32:51.283210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:32:51.284311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:32:51.284311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:32:51.284311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:32:51.284311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:32:51.284311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:51.284311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:51.284311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:51.284311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:51.284311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:51.284311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:51.284311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:51.284311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:51.284311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:51.284311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:51.284311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:51.284311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:51.285334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:51.285334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:51.285334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:51.285334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:51.285334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:32:51.285334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:32:51.285334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:32:51.285334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:32:51.285334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:32:51.285334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:32:51.285334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:32:51.285334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:32:51.285334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 580/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:32:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:51.290657 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cdc8d7-d9cf-4fec-909f-56b3039cdf5c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:51.299358 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:51.395140 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cdc8d7-d9cf-4fec-909f-56b3039cdf5c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:51.436322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207c0c7-b9a2-43ae-bc56-bbb9ae9de0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:51.436918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207c0c7-b9a2-43ae-bc56-bbb9ae9de0ce None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:51.437222 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207c0c7-b9a2-43ae-bc56-bbb9ae9de0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:51.437522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207c0c7-b9a2-43ae-bc56-bbb9ae9de0ce None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:51.438171 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207c0c7-b9a2-43ae-bc56-bbb9ae9de0ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:51.438476 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207c0c7-b9a2-43ae-bc56-bbb9ae9de0ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:51.439871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207c0c7-b9a2-43ae-bc56-bbb9ae9de0ce None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:51.448612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 577/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:32:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:51.455892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e27e5-9f33-40d5-878b-c49042a821a9 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:51.482983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:51.566124 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e27e5-9f33-40d5-878b-c49042a821a9 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:51.607276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65571130-b581-4389-9a4b-3effcbd40477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:51.607944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65571130-b581-4389-9a4b-3effcbd40477 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:51.608261 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65571130-b581-4389-9a4b-3effcbd40477 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:51.608532 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65571130-b581-4389-9a4b-3effcbd40477 None admin] PATH_INFO: `/v3/users/6f000ecf1679491cbc24f7900c0894ec` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:51.609793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65571130-b581-4389-9a4b-3effcbd40477 None admin] RBAC: Authorizing `identity:get_user(user_id=6f000ecf1679491cbc24f7900c0894ec)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:51.611293 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65571130-b581-4389-9a4b-3effcbd40477 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:51.612980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 581/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:32:51 2022] GET /identity/v3/users/6f000ecf1679491cbc24f7900c0894ec => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:32:51.619359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207c0c7-b9a2-43ae-bc56-bbb9ae9de0ce None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:51.692064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:51.726978 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207c0c7-b9a2-43ae-bc56-bbb9ae9de0ce None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:51.762529 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be5f3f-85b7-4271-992d-f3576d83f244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:51.763476 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be5f3f-85b7-4271-992d-f3576d83f244 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:51.763476 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be5f3f-85b7-4271-992d-f3576d83f244 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:51.763652 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be5f3f-85b7-4271-992d-f3576d83f244 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:51.834137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be5f3f-85b7-4271-992d-f3576d83f244 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:51.835607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be5f3f-85b7-4271-992d-f3576d83f244 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:51.910495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:51.947847 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 578/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:32:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:32:51.954611 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65571130-b581-4389-9a4b-3effcbd40477 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:52.053491 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:52.065183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65571130-b581-4389-9a4b-3effcbd40477 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:52.101533 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921f9ea-01cf-42fe-9223-4adcb42024b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:52.102161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921f9ea-01cf-42fe-9223-4adcb42024b2 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:52.102440 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921f9ea-01cf-42fe-9223-4adcb42024b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:52.102706 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921f9ea-01cf-42fe-9223-4adcb42024b2 None admin] PATH_INFO: `/v3/projects/23a2477dee1c4148bced7c591aaf6c9b` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:52.103886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921f9ea-01cf-42fe-9223-4adcb42024b2 None admin] RBAC: Authorizing `identity:get_project(project_id=23a2477dee1c4148bced7c591aaf6c9b)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:52.105361 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921f9ea-01cf-42fe-9223-4adcb42024b2 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:52.107274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 582/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:32:51 2022] GET /identity/v3/projects/23a2477dee1c4148bced7c591aaf6c9b => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:52.113839 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be5f3f-85b7-4271-992d-f3576d83f244 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:52.217229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be5f3f-85b7-4271-992d-f3576d83f244 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:52.252434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e586605-9a31-4c23-bd65-14f1bb1fd353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:52.253042 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e586605-9a31-4c23-bd65-14f1bb1fd353 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:52.253316 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e586605-9a31-4c23-bd65-14f1bb1fd353 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:52.253583 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e586605-9a31-4c23-bd65-14f1bb1fd353 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:52.254312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e586605-9a31-4c23-bd65-14f1bb1fd353 None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=23a2477dee1c4148bced7c591aaf6c9b, user.id=6f000ecf1679491cbc24f7900c0894ec) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:52.254585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e586605-9a31-4c23-bd65-14f1bb1fd353 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:52.256054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e586605-9a31-4c23-bd65-14f1bb1fd353 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:52.262333 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 579/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:32:52 2022] GET /identity/v3/role_assignments?user.id=6f000ecf1679491cbc24f7900c0894ec&scope.project.id=23a2477dee1c4148bced7c591aaf6c9b&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:54.301777 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbeb469-51fd-41be-9170-708fcfde0e8d None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:54.302352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbeb469-51fd-41be-9170-708fcfde0e8d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:54.302468 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbeb469-51fd-41be-9170-708fcfde0e8d None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:54.304658 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 583/1162] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:54.311208 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d18cb7-e585-4f1f-bd11-3fb7417daf12 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:54.311861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d18cb7-e585-4f1f-bd11-3fb7417daf12 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:54.311861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d18cb7-e585-4f1f-bd11-3fb7417daf12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:54.332337 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-33d18cb7-e585-4f1f-bd11-3fb7417daf12 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:54.333561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-33d18cb7-e585-4f1f-bd11-3fb7417daf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:54.362793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:54.414984 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 580/1163] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:54.687900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbeb469-51fd-41be-9170-708fcfde0e8d None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:54.807377 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbeb469-51fd-41be-9170-708fcfde0e8d None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:54.847926 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dad8d8-0175-42cc-8e6c-82fced105644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:54.848815 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dad8d8-0175-42cc-8e6c-82fced105644 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:54.849146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dad8d8-0175-42cc-8e6c-82fced105644 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:54.849577 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dad8d8-0175-42cc-8e6c-82fced105644 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:54.882882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:54.928146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dad8d8-0175-42cc-8e6c-82fced105644 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:54.929610 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dad8d8-0175-42cc-8e6c-82fced105644 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:55.038445 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 584/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:32:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:32:55.046016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d18cb7-e585-4f1f-bd11-3fb7417daf12 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:55.162110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d18cb7-e585-4f1f-bd11-3fb7417daf12 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:55.201761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6045359-36c7-428e-8b3f-15ee7ac92440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:55.202423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6045359-36c7-428e-8b3f-15ee7ac92440 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:55.202699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6045359-36c7-428e-8b3f-15ee7ac92440 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:55.202967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6045359-36c7-428e-8b3f-15ee7ac92440 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:55.211139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6045359-36c7-428e-8b3f-15ee7ac92440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:55.212669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6045359-36c7-428e-8b3f-15ee7ac92440 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:55.226632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-a6045359-36c7-428e-8b3f-15ee7ac92440 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 14:32:55.226632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:55.226632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 30 14:32:55.226632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 30 14:32:55.226632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:32:55.226632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:32:55.226632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:32:55.226632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:32:55.226632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 30 14:32:55.226632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:55.226632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:32:55.226632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:55.226632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:55.226632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:55.226632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:55.226632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:55.226632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:55.227924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:55.227924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:55.227924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:55.227924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:55.227924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:55.227924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:55.227924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:55.227924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:55.227924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:55.227924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:55.227924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:55.227924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:55.227924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:55.227924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:55.227924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:55.227924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:55.228854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:55.228854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:55.228854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:55.228854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:55.228854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:55.228854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:55.228854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:55.228854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:55.228854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:55.228854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:55.228854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 30 14:32:55.228854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:55.228854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 14:32:55.228854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:32:55.228854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 581/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Aug 30 14:32:55 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:55.235043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dad8d8-0175-42cc-8e6c-82fced105644 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:55.305444 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:55.354683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dad8d8-0175-42cc-8e6c-82fced105644 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:55.394233 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9587ae-fb86-4b42-b243-09f34fa7304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:55.394865 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9587ae-fb86-4b42-b243-09f34fa7304f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:55.395146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9587ae-fb86-4b42-b243-09f34fa7304f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:55.395403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9587ae-fb86-4b42-b243-09f34fa7304f None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:55.396111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9587ae-fb86-4b42-b243-09f34fa7304f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:55.396413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9587ae-fb86-4b42-b243-09f34fa7304f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:55.397844 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9587ae-fb86-4b42-b243-09f34fa7304f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:55.406566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 585/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Aug 30 14:32:55 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:57.308889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab1648-d3d8-4117-88d4-2610cc5d49d0 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:57.309490 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab1648-d3d8-4117-88d4-2610cc5d49d0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:57.309490 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab1648-d3d8-4117-88d4-2610cc5d49d0 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:57.311083 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 582/1167] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:57.316780 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a13be-126b-417b-8c08-832407102d05 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:57.317200 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a13be-126b-417b-8c08-832407102d05 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:57.317328 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a13be-126b-417b-8c08-832407102d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:57.338239 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-957a13be-126b-417b-8c08-832407102d05 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:57.339493 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-957a13be-126b-417b-8c08-832407102d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:57.416494 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 586/1168] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:57.690947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab1648-d3d8-4117-88d4-2610cc5d49d0 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:57.719430 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:57.813859 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab1648-d3d8-4117-88d4-2610cc5d49d0 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:57.852555 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954604e-c883-46e0-8212-d5828456e2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:57.853179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954604e-c883-46e0-8212-d5828456e2e5 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:57.853456 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954604e-c883-46e0-8212-d5828456e2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:57.853724 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954604e-c883-46e0-8212-d5828456e2e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:57.854426 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954604e-c883-46e0-8212-d5828456e2e5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:57.855954 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954604e-c883-46e0-8212-d5828456e2e5 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:57.876103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 583/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Tue Aug 30 14:32:57 2022] POST /identity/v3/domains => generated 267 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:32:59.863529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803a5b1-ed67-45ea-bb8e-e422bdde9c40 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:59.864304 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803a5b1-ed67-45ea-bb8e-e422bdde9c40 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:59.864304 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803a5b1-ed67-45ea-bb8e-e422bdde9c40 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:59.865615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 587/1170] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:59.872069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e84271-57e0-4ec6-a4be-f8ba0ba0b2e5 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:59.872693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e84271-57e0-4ec6-a4be-f8ba0ba0b2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:59.872693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e84271-57e0-4ec6-a4be-f8ba0ba0b2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:59.894392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-71e84271-57e0-4ec6-a4be-f8ba0ba0b2e5 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:59.895591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-71e84271-57e0-4ec6-a4be-f8ba0ba0b2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:59.983369 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 584/1171] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:33:00.260926 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803a5b1-ed67-45ea-bb8e-e422bdde9c40 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:00.320308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:00.373691 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803a5b1-ed67-45ea-bb8e-e422bdde9c40 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:00.416400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c15869-e99f-4733-8e33-affa4bbd812e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:00.417036 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c15869-e99f-4733-8e33-affa4bbd812e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:00.417312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c15869-e99f-4733-8e33-affa4bbd812e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:00.417582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c15869-e99f-4733-8e33-affa4bbd812e None admin] PATH_INFO: `/v3/domains/dd61565f2ac04da49753076f31aa9361` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:00.429979 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c15869-e99f-4733-8e33-affa4bbd812e None admin] RBAC: Authorizing `identity:get_domain(domain_id=dd61565f2ac04da49753076f31aa9361)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:00.431444 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c15869-e99f-4733-8e33-affa4bbd812e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:00.433164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 588/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Tue Aug 30 14:33:00 2022] GET /identity/v3/domains/dd61565f2ac04da49753076f31aa9361 => generated 267 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:00.439897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e84271-57e0-4ec6-a4be-f8ba0ba0b2e5 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:00.454857 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:00.545880 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e84271-57e0-4ec6-a4be-f8ba0ba0b2e5 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:00.580835 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b3439-264a-4220-be46-45512c5962aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:00.581459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b3439-264a-4220-be46-45512c5962aa None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:00.581740 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b3439-264a-4220-be46-45512c5962aa None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:00.582006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b3439-264a-4220-be46-45512c5962aa None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:00.582666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b3439-264a-4220-be46-45512c5962aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:00.584151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b3439-264a-4220-be46-45512c5962aa None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:00.614057 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 585/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Aug 30 14:33:00 2022] POST /identity/v3/projects => generated 361 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:33:02.554256 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aa3aa-6674-4fa7-b359-15114e416d42 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:02.554859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aa3aa-6674-4fa7-b359-15114e416d42 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:02.554859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aa3aa-6674-4fa7-b359-15114e416d42 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:02.556164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 589/1174] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:33:02.561499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f515ec0-2a4a-4905-97df-9b3182ab81ff None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:02.561939 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f515ec0-2a4a-4905-97df-9b3182ab81ff None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:02.562041 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f515ec0-2a4a-4905-97df-9b3182ab81ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:02.581936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-3f515ec0-2a4a-4905-97df-9b3182ab81ff None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:33:02.583131 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3f515ec0-2a4a-4905-97df-9b3182ab81ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:33:02.646207 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:02.657070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 586/1175] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:33:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:33:02.926642 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8aa3aa-6674-4fa7-b359-15114e416d42 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:03.044138 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8aa3aa-6674-4fa7-b359-15114e416d42 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:03.059131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:03.084968 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b035c-966e-4d35-86d5-6c5bd2b289a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:03.085596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b035c-966e-4d35-86d5-6c5bd2b289a0 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:03.085893 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b035c-966e-4d35-86d5-6c5bd2b289a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:03.086158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b035c-966e-4d35-86d5-6c5bd2b289a0 None admin] PATH_INFO: `/v3/domains/dd61565f2ac04da49753076f31aa9361` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:03.087341 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2b035c-966e-4d35-86d5-6c5bd2b289a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=dd61565f2ac04da49753076f31aa9361)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:03.088831 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2b035c-966e-4d35-86d5-6c5bd2b289a0 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:03.090559 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 590/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Tue Aug 30 14:33:02 2022] GET /identity/v3/domains/dd61565f2ac04da49753076f31aa9361 => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:03.097687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f515ec0-2a4a-4905-97df-9b3182ab81ff None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:03.218418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f515ec0-2a4a-4905-97df-9b3182ab81ff None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:03.256569 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97b28e-471b-4668-a3e5-f6fdd7608c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:03.257184 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97b28e-471b-4668-a3e5-f6fdd7608c87 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:03.257454 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97b28e-471b-4668-a3e5-f6fdd7608c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:03.257714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97b28e-471b-4668-a3e5-f6fdd7608c87 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:03.258372 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97b28e-471b-4668-a3e5-f6fdd7608c87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:03.259831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97b28e-471b-4668-a3e5-f6fdd7608c87 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:03.284091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 587/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:33:03 2022] POST /identity/v3/users => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:33:05.233326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f42ac1-addf-42d3-a55a-bd9b3c2fe7fc None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:05.233960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f42ac1-addf-42d3-a55a-bd9b3c2fe7fc None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:05.233960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f42ac1-addf-42d3-a55a-bd9b3c2fe7fc None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:05.235220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 591/1178] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:33:05.240717 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e553d96-5b17-4f6f-885f-5f06e12af530 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:05.241167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e553d96-5b17-4f6f-885f-5f06e12af530 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:05.241261 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e553d96-5b17-4f6f-885f-5f06e12af530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:05.260968 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7e553d96-5b17-4f6f-885f-5f06e12af530 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:33:05.262156 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7e553d96-5b17-4f6f-885f-5f06e12af530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:33:05.309779 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:05.334282 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 588/1179] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:33:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:33:05.622411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f42ac1-addf-42d3-a55a-bd9b3c2fe7fc None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:05.740855 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f42ac1-addf-42d3-a55a-bd9b3c2fe7fc None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:05.777730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a7b55-d962-41df-86c8-960954af1284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:05.778510 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a7b55-d962-41df-86c8-960954af1284 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:05.778782 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a7b55-d962-41df-86c8-960954af1284 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:05.779047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a7b55-d962-41df-86c8-960954af1284 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:05.789992 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a7b55-d962-41df-86c8-960954af1284 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:33:05.794793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-2b0a7b55-d962-41df-86c8-960954af1284 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:33:05.794793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:05.794793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:05.794793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:05.794793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:05.794793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:05.794793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:05.794793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:05.794793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:05.794793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:05.794793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:05.794793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:05.794793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:33:05.794793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:33:05.794793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:33:05.794793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:33:05.795838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:33:05.795838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:33:05.795838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:33:05.795838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:33:05.795838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:05.795838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:05.795838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:05.795838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:05.795838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:05.795838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:05.795838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:05.795838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:05.795838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:05.795838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:05.795838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:05.795838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:05.796837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:05.796837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:05.796837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:05.796837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:05.796837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:33:05.796837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:33:05.796837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:33:05.796837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:33:05.796837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:33:05.796837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:33:05.796837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:33:05.796837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:33:05.796837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 592/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:33:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:05.802360 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e553d96-5b17-4f6f-885f-5f06e12af530 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:05.903165 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e553d96-5b17-4f6f-885f-5f06e12af530 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:05.940055 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a8c21-624c-48a6-9e1d-fcacb5cf7986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:05.940678 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a8c21-624c-48a6-9e1d-fcacb5cf7986 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:05.940949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a8c21-624c-48a6-9e1d-fcacb5cf7986 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:05.941213 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a8c21-624c-48a6-9e1d-fcacb5cf7986 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:05.941871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a8c21-624c-48a6-9e1d-fcacb5cf7986 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:05.942169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a8c21-624c-48a6-9e1d-fcacb5cf7986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:05.943533 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a8c21-624c-48a6-9e1d-fcacb5cf7986 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:05.952546 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 589/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:33:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:05.959231 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a7b55-d962-41df-86c8-960954af1284 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:06.019885 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:06.088772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a7b55-d962-41df-86c8-960954af1284 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:06.122208 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7699a3-4e7e-4abc-a17c-830ae2c47122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:06.122842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7699a3-4e7e-4abc-a17c-830ae2c47122 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:06.123118 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7699a3-4e7e-4abc-a17c-830ae2c47122 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:06.123391 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7699a3-4e7e-4abc-a17c-830ae2c47122 None admin] PATH_INFO: `/v3/users/25040d20ffea4b07a18d92f0fadc9e22` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:06.138974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7699a3-4e7e-4abc-a17c-830ae2c47122 None admin] RBAC: Authorizing `identity:get_user(user_id=25040d20ffea4b07a18d92f0fadc9e22)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:06.140490 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7699a3-4e7e-4abc-a17c-830ae2c47122 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:06.142276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 593/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:33:05 2022] GET /identity/v3/users/25040d20ffea4b07a18d92f0fadc9e22 => generated 311 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:06.148931 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a8c21-624c-48a6-9e1d-fcacb5cf7986 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:06.199164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:06.266941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a8c21-624c-48a6-9e1d-fcacb5cf7986 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:06.306743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf20395-eeed-4175-9865-4e9d66b610b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:06.307370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf20395-eeed-4175-9865-4e9d66b610b2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:06.307650 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf20395-eeed-4175-9865-4e9d66b610b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:06.307913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf20395-eeed-4175-9865-4e9d66b610b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:06.383594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf20395-eeed-4175-9865-4e9d66b610b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:06.385086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf20395-eeed-4175-9865-4e9d66b610b2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:06.431248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:06.498846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 590/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:33:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:33:06.505039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7699a3-4e7e-4abc-a17c-830ae2c47122 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:06.603236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:06.607900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7699a3-4e7e-4abc-a17c-830ae2c47122 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:06.644433 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc7aa5-9d51-413f-bf99-12c349f4df0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:06.645067 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc7aa5-9d51-413f-bf99-12c349f4df0a None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:06.645339 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc7aa5-9d51-413f-bf99-12c349f4df0a None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:06.645601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc7aa5-9d51-413f-bf99-12c349f4df0a None admin] PATH_INFO: `/v3/projects/63555538b49e40e6a35686e649558511` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:06.646717 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc7aa5-9d51-413f-bf99-12c349f4df0a None admin] RBAC: Authorizing `identity:get_project(project_id=63555538b49e40e6a35686e649558511)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:06.648165 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc7aa5-9d51-413f-bf99-12c349f4df0a None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:06.649790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 594/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:33:06 2022] GET /identity/v3/projects/63555538b49e40e6a35686e649558511 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:33:06.656368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf20395-eeed-4175-9865-4e9d66b610b2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:06.761257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf20395-eeed-4175-9865-4e9d66b610b2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:06.797114 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef7b87-3aa2-4781-9e12-d52900b6fae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:06.797719 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef7b87-3aa2-4781-9e12-d52900b6fae0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:06.797992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef7b87-3aa2-4781-9e12-d52900b6fae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:06.798262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef7b87-3aa2-4781-9e12-d52900b6fae0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:06.798989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef7b87-3aa2-4781-9e12-d52900b6fae0 None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=63555538b49e40e6a35686e649558511, user.id=25040d20ffea4b07a18d92f0fadc9e22) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:06.799295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef7b87-3aa2-4781-9e12-d52900b6fae0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:06.800751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef7b87-3aa2-4781-9e12-d52900b6fae0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:06.806322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 591/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:33:06 2022] GET /identity/v3/role_assignments?user.id=25040d20ffea4b07a18d92f0fadc9e22&scope.project.id=63555538b49e40e6a35686e649558511&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:08.678002 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc230c-ecb8-4637-9bb9-d1950036abf8 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:08.678588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc230c-ecb8-4637-9bb9-d1950036abf8 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:08.678588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc230c-ecb8-4637-9bb9-d1950036abf8 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:08.679896 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 595/1186] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:33:08.685725 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a32665-5515-4631-965e-a70bbb09fdbf None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:08.686218 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a32665-5515-4631-965e-a70bbb09fdbf None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:08.686340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a32665-5515-4631-965e-a70bbb09fdbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:08.699886 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:08.707103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e1a32665-5515-4631-965e-a70bbb09fdbf None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:33:08.708375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e1a32665-5515-4631-965e-a70bbb09fdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:33:08.783493 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 592/1187] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:33:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:33:09.047970 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc230c-ecb8-4637-9bb9-d1950036abf8 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:09.154401 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc230c-ecb8-4637-9bb9-d1950036abf8 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:09.189853 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919616c-9b6a-43bf-8aa6-0bbb2de07880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:09.190492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919616c-9b6a-43bf-8aa6-0bbb2de07880 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:09.190764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919616c-9b6a-43bf-8aa6-0bbb2de07880 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:09.191028 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919616c-9b6a-43bf-8aa6-0bbb2de07880 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:09.201960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919616c-9b6a-43bf-8aa6-0bbb2de07880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:33:09.207078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-2919616c-9b6a-43bf-8aa6-0bbb2de07880 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:33:09.207078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:09.207078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:09.207078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:09.207078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:09.207078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:09.207078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:09.207078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:09.207078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:09.207078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:09.207078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:09.207078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:09.207078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:33:09.207078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:33:09.207078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:33:09.207078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:33:09.208161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:33:09.208161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:33:09.208161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:33:09.208161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:33:09.208161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:09.208161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:09.208161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:09.208161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:09.208161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:09.208161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:09.208161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:09.208161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:09.208161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:09.208161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:09.208161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:09.208161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:09.209012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:09.209012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:09.209012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:09.209012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:09.209012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:33:09.209012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:33:09.209012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:33:09.209012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:33:09.209012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:33:09.209012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:33:09.209012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:33:09.209012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:33:09.209012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 596/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:33:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:09.214567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a32665-5515-4631-965e-a70bbb09fdbf None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:09.317638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a32665-5515-4631-965e-a70bbb09fdbf None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:09.354899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50793d-3330-4206-976d-fa4d9e0662fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:09.355507 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50793d-3330-4206-976d-fa4d9e0662fb None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:09.355780 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50793d-3330-4206-976d-fa4d9e0662fb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:09.356150 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50793d-3330-4206-976d-fa4d9e0662fb None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:09.356839 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50793d-3330-4206-976d-fa4d9e0662fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:09.357133 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50793d-3330-4206-976d-fa4d9e0662fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:09.358477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50793d-3330-4206-976d-fa4d9e0662fb None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:09.360910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:09.368840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 593/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:33:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:09.376189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919616c-9b6a-43bf-8aa6-0bbb2de07880 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:09.412031 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:09.495812 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919616c-9b6a-43bf-8aa6-0bbb2de07880 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:09.535199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6837f-cdf6-48b1-9d63-3d73c86464b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:09.535850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6837f-cdf6-48b1-9d63-3d73c86464b2 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:09.536149 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6837f-cdf6-48b1-9d63-3d73c86464b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:09.536413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6837f-cdf6-48b1-9d63-3d73c86464b2 None admin] PATH_INFO: `/v3/users/25040d20ffea4b07a18d92f0fadc9e22` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:09.537635 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6837f-cdf6-48b1-9d63-3d73c86464b2 None admin] RBAC: Authorizing `identity:get_user(user_id=25040d20ffea4b07a18d92f0fadc9e22)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:09.539080 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6837f-cdf6-48b1-9d63-3d73c86464b2 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:09.540747 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 597/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:33:09 2022] GET /identity/v3/users/25040d20ffea4b07a18d92f0fadc9e22 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:09.547214 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50793d-3330-4206-976d-fa4d9e0662fb None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:09.661536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50793d-3330-4206-976d-fa4d9e0662fb None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:09.698701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad8873-87b5-4970-8490-debad2b6e32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:09.699394 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad8873-87b5-4970-8490-debad2b6e32a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:09.699593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad8873-87b5-4970-8490-debad2b6e32a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:09.700159 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad8873-87b5-4970-8490-debad2b6e32a None admin] PATH_INFO: `/v3/projects/63555538b49e40e6a35686e649558511` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:09.701420 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad8873-87b5-4970-8490-debad2b6e32a None admin] RBAC: Authorizing `identity:get_project(project_id=63555538b49e40e6a35686e649558511)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:09.702898 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad8873-87b5-4970-8490-debad2b6e32a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:09.704576 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 594/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:33:09 2022] GET /identity/v3/projects/63555538b49e40e6a35686e649558511 => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:09.712083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6837f-cdf6-48b1-9d63-3d73c86464b2 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:09.808690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:09.816706 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6837f-cdf6-48b1-9d63-3d73c86464b2 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:09.854959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88898b7b-065d-4ebe-8aed-ec6c7108d798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:09.855621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88898b7b-065d-4ebe-8aed-ec6c7108d798 None admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:09.855877 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88898b7b-065d-4ebe-8aed-ec6c7108d798 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:09.856144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88898b7b-065d-4ebe-8aed-ec6c7108d798 None admin] PATH_INFO: `/v3/projects/63555538b49e40e6a35686e649558511/users/25040d20ffea4b07a18d92f0fadc9e22/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:09.858067 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88898b7b-065d-4ebe-8aed-ec6c7108d798 None admin] RBAC: Authorizing `identity:create_grant(project_id=63555538b49e40e6a35686e649558511, user_id=25040d20ffea4b07a18d92f0fadc9e22, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:09.859487 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88898b7b-065d-4ebe-8aed-ec6c7108d798 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:09.873388 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 598/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:33:09 2022] PUT /identity/v3/projects/63555538b49e40e6a35686e649558511/users/25040d20ffea4b07a18d92f0fadc9e22/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:33:11.744369 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e822f7-8ace-4517-924f-ef31ea960eff None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:11.744932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e822f7-8ace-4517-924f-ef31ea960eff None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:11.744932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e822f7-8ace-4517-924f-ef31ea960eff None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:11.746254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 595/1193] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:33:11.752723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed345bb8-d871-4089-984e-114b9f99199e None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:11.753227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed345bb8-d871-4089-984e-114b9f99199e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:11.753227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed345bb8-d871-4089-984e-114b9f99199e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:11.773416 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-ed345bb8-d871-4089-984e-114b9f99199e None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:33:11.774630 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-ed345bb8-d871-4089-984e-114b9f99199e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:33:11.845080 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 599/1194] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:33:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:33:12.107662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e822f7-8ace-4517-924f-ef31ea960eff None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:12.178181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:12.216846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e822f7-8ace-4517-924f-ef31ea960eff None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:12.252323 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e2701-aaae-4a35-90bb-99e9efee3aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:12.252947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e2701-aaae-4a35-90bb-99e9efee3aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:12.253215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e2701-aaae-4a35-90bb-99e9efee3aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:12.253479 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e2701-aaae-4a35-90bb-99e9efee3aa9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:12.264480 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e2701-aaae-4a35-90bb-99e9efee3aa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:33:12.269494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-411e2701-aaae-4a35-90bb-99e9efee3aa9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:33:12.269494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:12.269494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:12.269494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:12.269494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:12.269494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:12.269494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:12.269494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:12.269494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:12.269494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:12.269494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:12.269494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:12.269494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:33:12.269494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:33:12.269494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:33:12.269494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:33:12.270585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:33:12.270585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:33:12.270585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:33:12.270585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:33:12.270585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:12.270585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:12.270585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:12.270585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:12.270585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:12.270585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:12.270585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:12.270585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:12.270585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:12.270585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:12.270585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:12.270585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:12.271689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:12.271689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:12.271689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:12.271689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:12.271689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:33:12.271689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:33:12.271689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:33:12.271689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:33:12.271689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:33:12.271689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:33:12.271689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:33:12.271689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:33:12.275617 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 596/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:33:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:12.276854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed345bb8-d871-4089-984e-114b9f99199e None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:12.367371 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:12.384171 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed345bb8-d871-4089-984e-114b9f99199e None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:12.421585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a41de3-65de-4877-8650-b0fbbb282813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:12.422211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a41de3-65de-4877-8650-b0fbbb282813 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:12.422489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a41de3-65de-4877-8650-b0fbbb282813 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:12.422754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a41de3-65de-4877-8650-b0fbbb282813 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:12.423432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a41de3-65de-4877-8650-b0fbbb282813 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:12.423733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a41de3-65de-4877-8650-b0fbbb282813 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:12.425138 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a41de3-65de-4877-8650-b0fbbb282813 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:12.434083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 600/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:33:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:12.440645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e2701-aaae-4a35-90bb-99e9efee3aa9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:12.551795 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e2701-aaae-4a35-90bb-99e9efee3aa9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:12.580676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:12.591581 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca78b1-6c41-4d3f-9398-5a4950a4dfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:12.592287 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca78b1-6c41-4d3f-9398-5a4950a4dfdf None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:12.592561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca78b1-6c41-4d3f-9398-5a4950a4dfdf None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:12.592812 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca78b1-6c41-4d3f-9398-5a4950a4dfdf None admin] PATH_INFO: `/v3/users/25040d20ffea4b07a18d92f0fadc9e22` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:12.594051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca78b1-6c41-4d3f-9398-5a4950a4dfdf None admin] RBAC: Authorizing `identity:get_user(user_id=25040d20ffea4b07a18d92f0fadc9e22)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:12.595522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca78b1-6c41-4d3f-9398-5a4950a4dfdf None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:12.597222 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 597/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Aug 30 14:33:12 2022] GET /identity/v3/users/25040d20ffea4b07a18d92f0fadc9e22 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:12.603687 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a41de3-65de-4877-8650-b0fbbb282813 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:12.716725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a41de3-65de-4877-8650-b0fbbb282813 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:12.754478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf95a8-0e56-4a02-8371-c9b60debfc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:12.755229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf95a8-0e56-4a02-8371-c9b60debfc38 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:12.755324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf95a8-0e56-4a02-8371-c9b60debfc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:12.755619 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf95a8-0e56-4a02-8371-c9b60debfc38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:12.772881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:12.827479 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf95a8-0e56-4a02-8371-c9b60debfc38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:12.828974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf95a8-0e56-4a02-8371-c9b60debfc38 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:12.935628 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 601/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:33:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:33:12.941816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca78b1-6c41-4d3f-9398-5a4950a4dfdf None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:13.049099 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca78b1-6c41-4d3f-9398-5a4950a4dfdf None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:13.085638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed85e58-6d23-4f17-9af9-a2f77f51530c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:13.086266 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed85e58-6d23-4f17-9af9-a2f77f51530c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:13.086536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed85e58-6d23-4f17-9af9-a2f77f51530c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:13.086801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed85e58-6d23-4f17-9af9-a2f77f51530c None admin] PATH_INFO: `/v3/projects/63555538b49e40e6a35686e649558511` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:13.087966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed85e58-6d23-4f17-9af9-a2f77f51530c None admin] RBAC: Authorizing `identity:get_project(project_id=63555538b49e40e6a35686e649558511)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:13.089428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed85e58-6d23-4f17-9af9-a2f77f51530c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:13.091101 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 598/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Aug 30 14:33:12 2022] GET /identity/v3/projects/63555538b49e40e6a35686e649558511 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:13.097505 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf95a8-0e56-4a02-8371-c9b60debfc38 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:13.148365 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:13.206516 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf95a8-0e56-4a02-8371-c9b60debfc38 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:13.244284 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14525bf-9276-4e1d-b95c-d3cfffcf11c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:13.244890 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14525bf-9276-4e1d-b95c-d3cfffcf11c5 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:13.245160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14525bf-9276-4e1d-b95c-d3cfffcf11c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:13.245421 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14525bf-9276-4e1d-b95c-d3cfffcf11c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:13.246137 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14525bf-9276-4e1d-b95c-d3cfffcf11c5 None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=63555538b49e40e6a35686e649558511, user.id=25040d20ffea4b07a18d92f0fadc9e22) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:13.246439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14525bf-9276-4e1d-b95c-d3cfffcf11c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:13.248145 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14525bf-9276-4e1d-b95c-d3cfffcf11c5 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:13.254175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 602/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:33:13 2022] GET /identity/v3/role_assignments?user.id=25040d20ffea4b07a18d92f0fadc9e22&scope.project.id=63555538b49e40e6a35686e649558511&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:23.080783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=69 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:23.251559 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=45 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:11.283747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fb2ab-ee34-475c-bd8d-91debf53f66b None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:11.284439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fb2ab-ee34-475c-bd8d-91debf53f66b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:11.284595 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fb2ab-ee34-475c-bd8d-91debf53f66b None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:11.286862 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 599/1201] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:34:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:11.297311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb2a69-8abe-4ad1-a36d-50fe1282709e None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:11.297311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb2a69-8abe-4ad1-a36d-50fe1282709e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:11.297311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb2a69-8abe-4ad1-a36d-50fe1282709e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:11.320807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-7efb2a69-8abe-4ad1-a36d-50fe1282709e None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:11.322451 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7efb2a69-8abe-4ad1-a36d-50fe1282709e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:11.433331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 603/1202] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:34:11.735533 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fb2ab-ee34-475c-bd8d-91debf53f66b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:11.886501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fb2ab-ee34-475c-bd8d-91debf53f66b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:11.936253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953fbbc-6772-4ca8-87d1-fbb78688ddd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:11.936830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953fbbc-6772-4ca8-87d1-fbb78688ddd2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:11.937128 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953fbbc-6772-4ca8-87d1-fbb78688ddd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:11.937406 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953fbbc-6772-4ca8-87d1-fbb78688ddd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:11.951316 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953fbbc-6772-4ca8-87d1-fbb78688ddd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:11.953104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953fbbc-6772-4ca8-87d1-fbb78688ddd2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:11.981962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-f953fbbc-6772-4ca8-87d1-fbb78688ddd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:11.981962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:11.981962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:11.981962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:11.981962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:11.981962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:11.981962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:11.981962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:11.981962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:11.981962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:11.981962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:11.981962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:11.981962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:34:11.981962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:34:11.981962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:34:11.981962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:34:11.983783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:11.983783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:11.983783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:11.983783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:11.983783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:11.983783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:11.983783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:11.983783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:11.983783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:11.983783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:11.983783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:11.983783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:11.983783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:11.983783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:11.983783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:11.983783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:11.983783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:34:11.984941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:34:11.984941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:11.984941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:34:11.984941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 600/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:34:11.990195 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb2a69-8abe-4ad1-a36d-50fe1282709e None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:12.160745 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb2a69-8abe-4ad1-a36d-50fe1282709e None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:12.208263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:12.223819 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b425a2-e118-436f-96cc-facbc0905211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:12.224569 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b425a2-e118-436f-96cc-facbc0905211 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:12.224876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b425a2-e118-436f-96cc-facbc0905211 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:12.225117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b425a2-e118-436f-96cc-facbc0905211 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:12.225820 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b425a2-e118-436f-96cc-facbc0905211 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:12.226163 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b425a2-e118-436f-96cc-facbc0905211 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:12.227647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b425a2-e118-436f-96cc-facbc0905211 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:12.242445 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 604/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:12.253181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953fbbc-6772-4ca8-87d1-fbb78688ddd2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:12.361642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:12.447449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953fbbc-6772-4ca8-87d1-fbb78688ddd2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:12.496893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8479e-a27a-4a2f-987e-b7e82e8e9116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:12.497899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8479e-a27a-4a2f-987e-b7e82e8e9116 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:12.498328 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8479e-a27a-4a2f-987e-b7e82e8e9116 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:12.498816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8479e-a27a-4a2f-987e-b7e82e8e9116 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:12.499687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8479e-a27a-4a2f-987e-b7e82e8e9116 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:12.501796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8479e-a27a-4a2f-987e-b7e82e8e9116 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:12.533031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 601/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Aug 30 14:34:12 2022] POST /identity/v3/users => generated 251 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:34:14.756176 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363cfd4-6ce4-4dec-9db7-c059cfd1a26c None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:14.756176 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363cfd4-6ce4-4dec-9db7-c059cfd1a26c None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:14.756176 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363cfd4-6ce4-4dec-9db7-c059cfd1a26c None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:14.756176 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 605/1206] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:14.760727 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0ae40-7f27-4446-a8eb-041e15cac53c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:14.761239 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0ae40-7f27-4446-a8eb-041e15cac53c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:14.761239 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0ae40-7f27-4446-a8eb-041e15cac53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:14.781457 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-73e0ae40-7f27-4446-a8eb-041e15cac53c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:14.783032 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-73e0ae40-7f27-4446-a8eb-041e15cac53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:14.868081 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 602/1207] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:34:15.158636 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363cfd4-6ce4-4dec-9db7-c059cfd1a26c None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:15.310546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363cfd4-6ce4-4dec-9db7-c059cfd1a26c None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:15.369401 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b98cb5-78ee-4c50-bf0c-669fee06e623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:15.370139 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b98cb5-78ee-4c50-bf0c-669fee06e623 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:15.370413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b98cb5-78ee-4c50-bf0c-669fee06e623 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:15.370902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b98cb5-78ee-4c50-bf0c-669fee06e623 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:15.388806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b98cb5-78ee-4c50-bf0c-669fee06e623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:34:15.394587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-72b98cb5-78ee-4c50-bf0c-669fee06e623 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:34:15.394587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:15.394587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:15.394587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:15.394587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:15.394587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:15.394587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:15.394587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:15.394587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:15.394587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:15.394587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:15.394587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:15.394587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:34:15.394587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:34:15.394587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:34:15.394587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:34:15.395614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:34:15.395614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:34:15.395614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:34:15.395614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:34:15.395614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:15.395614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:15.395614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:15.395614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:15.395614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:15.395614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:15.395614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:15.395614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:15.395614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:15.395614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:15.395614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:15.395614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:15.396507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:15.396507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:15.396507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:15.396507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:15.396507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:34:15.396507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:34:15.396507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:34:15.396507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:34:15.396507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:34:15.396507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:34:15.396507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:34:15.396507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:15.396507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 606/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:34:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:15.403090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0ae40-7f27-4446-a8eb-041e15cac53c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:15.459272 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:15.558852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0ae40-7f27-4446-a8eb-041e15cac53c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:15.604040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44b8dc-cb87-4c5a-9190-7b2027db5932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:15.605021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44b8dc-cb87-4c5a-9190-7b2027db5932 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:15.605469 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44b8dc-cb87-4c5a-9190-7b2027db5932 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:15.605905 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44b8dc-cb87-4c5a-9190-7b2027db5932 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:15.606750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44b8dc-cb87-4c5a-9190-7b2027db5932 None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:15.607099 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44b8dc-cb87-4c5a-9190-7b2027db5932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:15.608535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44b8dc-cb87-4c5a-9190-7b2027db5932 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:15.617724 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 603/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:34:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:15.624610 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b98cb5-78ee-4c50-bf0c-669fee06e623 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:15.692368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:15.766415 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b98cb5-78ee-4c50-bf0c-669fee06e623 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:15.811854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719add0-7728-480f-a812-3086bf7cf4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:15.812728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719add0-7728-480f-a812-3086bf7cf4af None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:15.813083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719add0-7728-480f-a812-3086bf7cf4af None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:15.813511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719add0-7728-480f-a812-3086bf7cf4af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:15.831095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719add0-7728-480f-a812-3086bf7cf4af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:15.833238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719add0-7728-480f-a812-3086bf7cf4af None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:15.857095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-5719add0-7728-480f-a812-3086bf7cf4af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:15.857095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:15.857095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:15.857095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:15.857095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:15.857095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:15.857095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:15.857095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:15.857095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:15.857095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:15.857095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:15.857095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:15.857095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:34:15.857095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:34:15.857095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:34:15.857095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:34:15.859166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:15.859166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:15.859166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:15.859166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:15.859166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:15.859166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:15.859166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:15.859166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:15.859166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:15.859166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:15.859166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:15.859166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:15.859166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:15.859166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:15.859166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:15.859166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:15.859166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:34:15.860346 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:34:15.860346 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:15.860346 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:15.860346 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 607/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:15.865611 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44b8dc-cb87-4c5a-9190-7b2027db5932 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:15.992065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:15.995699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44b8dc-cb87-4c5a-9190-7b2027db5932 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:16.040619 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65188fb0-76b4-46e6-b346-8ca37ef6ec7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:16.040840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65188fb0-76b4-46e6-b346-8ca37ef6ec7b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:16.041105 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65188fb0-76b4-46e6-b346-8ca37ef6ec7b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:16.041352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65188fb0-76b4-46e6-b346-8ca37ef6ec7b None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:16.042289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65188fb0-76b4-46e6-b346-8ca37ef6ec7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:16.042780 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65188fb0-76b4-46e6-b346-8ca37ef6ec7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:16.044711 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65188fb0-76b4-46e6-b346-8ca37ef6ec7b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:16.059641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 604/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:34:16.065498 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719add0-7728-480f-a812-3086bf7cf4af None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:16.190405 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719add0-7728-480f-a812-3086bf7cf4af None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:16.192830 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:16.250982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87fb8a-c7f7-4574-ad48-affbb61381e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:16.251769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87fb8a-c7f7-4574-ad48-affbb61381e1 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:16.252122 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87fb8a-c7f7-4574-ad48-affbb61381e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:16.252386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87fb8a-c7f7-4574-ad48-affbb61381e1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:16.261794 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e87fb8a-c7f7-4574-ad48-affbb61381e1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:16.263839 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e87fb8a-c7f7-4574-ad48-affbb61381e1 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:16.277019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-2e87fb8a-c7f7-4574-ad48-affbb61381e1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:34:16.277019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:16.277019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:16.277019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:16.277019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:16.277019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:16.277019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:16.277019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:16.277019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:16.277019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:16.277019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:16.277019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:16.277019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:34:16.277019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:34:16.277019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:34:16.277019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:16.289211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:16.291615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:16.291615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:16.291615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:34:16.291615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:34:16.291615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:34:16.291615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:34:16.291615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:34:16.291615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:34:16.291615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:34:16.291615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:16.291615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 608/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:16.293113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65188fb0-76b4-46e6-b346-8ca37ef6ec7b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:16.442086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65188fb0-76b4-46e6-b346-8ca37ef6ec7b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:16.496658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5e7f8-a875-40c7-85f4-7f31af4d4039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:16.497222 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5e7f8-a875-40c7-85f4-7f31af4d4039 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:16.497222 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5e7f8-a875-40c7-85f4-7f31af4d4039 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:16.497497 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5e7f8-a875-40c7-85f4-7f31af4d4039 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:16.498381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a5e7f8-a875-40c7-85f4-7f31af4d4039 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:16.498662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a5e7f8-a875-40c7-85f4-7f31af4d4039 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:16.500438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a5e7f8-a875-40c7-85f4-7f31af4d4039 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:16.531099 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 605/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Aug 30 14:34:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:34:16.537441 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87fb8a-c7f7-4574-ad48-affbb61381e1 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:16.676680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87fb8a-c7f7-4574-ad48-affbb61381e1 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:16.726109 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c48951-fcd5-4554-9751-0384e5025c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:16.727112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c48951-fcd5-4554-9751-0384e5025c88 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:16.727582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c48951-fcd5-4554-9751-0384e5025c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:16.728017 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c48951-fcd5-4554-9751-0384e5025c88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:16.779004 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:16.843065 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c48951-fcd5-4554-9751-0384e5025c88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:16.845096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c48951-fcd5-4554-9751-0384e5025c88 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:16.980879 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 609/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:34:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:34:16.987604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5e7f8-a875-40c7-85f4-7f31af4d4039 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:17.022620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:17.129493 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5e7f8-a875-40c7-85f4-7f31af4d4039 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:17.176531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9a653-48bc-4ad6-bd73-e6ce38cb6b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:17.177269 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9a653-48bc-4ad6-bd73-e6ce38cb6b34 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:17.177512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9a653-48bc-4ad6-bd73-e6ce38cb6b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:17.177836 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9a653-48bc-4ad6-bd73-e6ce38cb6b34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:17.193206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9a653-48bc-4ad6-bd73-e6ce38cb6b34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:17.195073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9a653-48bc-4ad6-bd73-e6ce38cb6b34 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:17.215190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-8ec9a653-48bc-4ad6-bd73-e6ce38cb6b34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:17.215190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:17.215190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:17.215190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:17.215190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:17.215190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:17.215190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:17.215190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:17.215190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:17.215190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:17.215190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:17.215190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:17.215190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:34:17.215190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:34:17.215190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:34:17.215190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:34:17.217733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:17.217733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:17.217733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:17.217733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:17.217733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:17.217733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:17.217733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:17.217733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:17.217733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:17.217733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:17.217733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:17.217733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:17.217733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:17.217733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:17.217733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:17.217733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:17.217733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:34:17.219275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:34:17.219275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:17.219275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:34:17.219275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 606/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:17.223321 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c48951-fcd5-4554-9751-0384e5025c88 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:17.335727 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:17.400316 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c48951-fcd5-4554-9751-0384e5025c88 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:17.438940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853fa70e-9119-4305-8766-d85994858620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:17.439634 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853fa70e-9119-4305-8766-d85994858620 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:17.439949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853fa70e-9119-4305-8766-d85994858620 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:17.440247 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853fa70e-9119-4305-8766-d85994858620 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:17.440980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853fa70e-9119-4305-8766-d85994858620 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:17.441259 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853fa70e-9119-4305-8766-d85994858620 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:17.442854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853fa70e-9119-4305-8766-d85994858620 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:17.461525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 610/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:34:17.463724 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9a653-48bc-4ad6-bd73-e6ce38cb6b34 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:17.570220 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:17.609281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9a653-48bc-4ad6-bd73-e6ce38cb6b34 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:17.655533 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df22d4d-3700-40dd-9081-a698e03937cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:17.656260 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df22d4d-3700-40dd-9081-a698e03937cf None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:17.656537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df22d4d-3700-40dd-9081-a698e03937cf None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:17.656802 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df22d4d-3700-40dd-9081-a698e03937cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:17.664866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df22d4d-3700-40dd-9081-a698e03937cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:17.666338 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df22d4d-3700-40dd-9081-a698e03937cf None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:17.680455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-6df22d4d-3700-40dd-9081-a698e03937cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:34:17.680455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:17.680455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:17.680455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:17.680455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:17.680455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:17.680455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:17.680455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:17.680455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:17.680455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:17.680455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:17.680455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:17.680455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:34:17.680455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:34:17.680455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:34:17.681594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:34:17.681594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:17.681594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:17.681594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:17.681594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:17.681594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:17.681594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:17.681594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:17.681594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:17.681594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:17.681594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:17.681594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:17.681594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:17.681594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:17.681594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:17.681594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:17.681594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:17.687708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:34:17.687708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:34:17.687708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:34:17.687708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:34:17.687708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:34:17.687708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:34:17.687708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:34:17.687708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:34:17.687708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 607/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:34:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:17.688817 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853fa70e-9119-4305-8766-d85994858620 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:17.804076 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853fa70e-9119-4305-8766-d85994858620 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:17.827247 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:17.841326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bbe329-c238-4e13-96a7-2030778db1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:17.841746 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbe329-c238-4e13-96a7-2030778db1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:17.842022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbe329-c238-4e13-96a7-2030778db1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:17.842287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbe329-c238-4e13-96a7-2030778db1f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:17.842987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bbe329-c238-4e13-96a7-2030778db1f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:17.843289 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bbe329-c238-4e13-96a7-2030778db1f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:17.844758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bbe329-c238-4e13-96a7-2030778db1f6 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:17.858614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 611/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:34:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:17.865418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df22d4d-3700-40dd-9081-a698e03937cf None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:17.977061 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df22d4d-3700-40dd-9081-a698e03937cf None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:18.006744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:18.014604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d1421-6a75-496a-afc8-232403219993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:18.015464 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d1421-6a75-496a-afc8-232403219993 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:18.015952 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d1421-6a75-496a-afc8-232403219993 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:18.016578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d1421-6a75-496a-afc8-232403219993 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:18.017865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d1421-6a75-496a-afc8-232403219993 None admin] RBAC: Adding query filter params (role.id=f0fd911d7c9246aeb3d67299473a9f42, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=d7cdd6b1ee9f4e97beb9bb037e446c8f) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:18.018459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d1421-6a75-496a-afc8-232403219993 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:18.020079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d1421-6a75-496a-afc8-232403219993 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:18.026718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 608/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:34:17 2022] GET /identity/v3/role_assignments?user.id=d7cdd6b1ee9f4e97beb9bb037e446c8f&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=f0fd911d7c9246aeb3d67299473a9f42 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:20.295654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d1347-2b42-4f60-a777-874b8b411e75 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:20.296907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d1347-2b42-4f60-a777-874b8b411e75 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:20.296907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d1347-2b42-4f60-a777-874b8b411e75 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:20.297605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 612/1220] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:20.304317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12324d75-225f-4af7-967b-b9a0070ec4c4 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:20.304895 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12324d75-225f-4af7-967b-b9a0070ec4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:20.304895 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12324d75-225f-4af7-967b-b9a0070ec4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:20.325246 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-12324d75-225f-4af7-967b-b9a0070ec4c4 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:20.326457 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-12324d75-225f-4af7-967b-b9a0070ec4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:20.403537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 609/1221] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:34:20.674993 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d1347-2b42-4f60-a777-874b8b411e75 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:20.787483 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d1347-2b42-4f60-a777-874b8b411e75 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:20.830053 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61f6fa-c53d-4ffa-b840-ee0ad6c0c37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:20.830688 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61f6fa-c53d-4ffa-b840-ee0ad6c0c37d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:20.830983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61f6fa-c53d-4ffa-b840-ee0ad6c0c37d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:20.831230 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61f6fa-c53d-4ffa-b840-ee0ad6c0c37d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:20.842925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc61f6fa-c53d-4ffa-b840-ee0ad6c0c37d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:34:20.847730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-bc61f6fa-c53d-4ffa-b840-ee0ad6c0c37d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:34:20.847730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:20.847730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:20.847730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:20.847730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:20.847730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:20.847730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:20.847730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:20.847730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:20.847730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:20.847730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:20.847730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:20.847730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:34:20.847730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:34:20.847730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:34:20.847730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:34:20.848829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:34:20.848829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:34:20.848829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:34:20.848829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:34:20.848829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:20.848829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:20.848829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:20.848829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:20.848829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:20.848829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:20.848829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:20.848829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:20.848829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:20.848829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:20.848829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:20.848829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:20.849897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:20.849897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:20.849897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:20.849897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:20.849897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:34:20.849897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:34:20.849897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:34:20.849897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:34:20.849897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:34:20.849897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:34:20.849897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:34:20.849897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:20.852317 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 613/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:34:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:20.855858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12324d75-225f-4af7-967b-b9a0070ec4c4 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:20.962583 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12324d75-225f-4af7-967b-b9a0070ec4c4 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:20.987167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:20.998816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cdac55-2725-4c47-ac96-b2b9a13786ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:20.999502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdac55-2725-4c47-ac96-b2b9a13786ef None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:20.999801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdac55-2725-4c47-ac96-b2b9a13786ef None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:21.000129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdac55-2725-4c47-ac96-b2b9a13786ef None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:21.000935 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cdac55-2725-4c47-ac96-b2b9a13786ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:21.001293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cdac55-2725-4c47-ac96-b2b9a13786ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:21.002858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cdac55-2725-4c47-ac96-b2b9a13786ef None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:21.012200 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 610/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:34:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:21.018484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61f6fa-c53d-4ffa-b840-ee0ad6c0c37d None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:21.064097 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:21.127675 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61f6fa-c53d-4ffa-b840-ee0ad6c0c37d None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:21.169607 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ccf87c-1384-4f51-a355-4285221326a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:21.170253 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccf87c-1384-4f51-a355-4285221326a0 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:21.170535 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccf87c-1384-4f51-a355-4285221326a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:21.170795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccf87c-1384-4f51-a355-4285221326a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:21.181639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ccf87c-1384-4f51-a355-4285221326a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:21.183616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ccf87c-1384-4f51-a355-4285221326a0 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:21.199543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-28ccf87c-1384-4f51-a355-4285221326a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:21.199543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:21.199543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:21.199543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:21.199543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:21.199543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:21.199543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:21.199543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:21.199543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:21.199543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:21.199543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:21.199543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:21.199543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:34:21.199543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:34:21.199543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:34:21.199543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:34:21.200684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:21.200684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:21.200684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:21.200684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:21.200684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:21.200684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:21.200684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:21.200684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:21.200684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:21.200684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:21.200684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:21.200684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:21.200684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:21.200684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:21.200684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:21.200684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:21.200684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:34:21.201628 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:34:21.201628 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:21.201628 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:21.201628 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 614/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:21.207706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cdac55-2725-4c47-ac96-b2b9a13786ef None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:21.327373 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cdac55-2725-4c47-ac96-b2b9a13786ef None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:21.373288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331684ce-4b3a-4256-a6bd-877c79e5e3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:21.373999 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331684ce-4b3a-4256-a6bd-877c79e5e3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:21.374351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331684ce-4b3a-4256-a6bd-877c79e5e3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:21.374651 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331684ce-4b3a-4256-a6bd-877c79e5e3e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:21.375375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331684ce-4b3a-4256-a6bd-877c79e5e3e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:21.375686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331684ce-4b3a-4256-a6bd-877c79e5e3e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:21.377381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331684ce-4b3a-4256-a6bd-877c79e5e3e8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:21.390909 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 611/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:21.397331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ccf87c-1384-4f51-a355-4285221326a0 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:21.504812 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:21.514071 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ccf87c-1384-4f51-a355-4285221326a0 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:21.565951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7006847-711c-497c-8a6c-04cdea8f653c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:21.566864 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7006847-711c-497c-8a6c-04cdea8f653c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:21.567196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7006847-711c-497c-8a6c-04cdea8f653c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:21.567525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7006847-711c-497c-8a6c-04cdea8f653c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:21.577459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7006847-711c-497c-8a6c-04cdea8f653c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:21.578872 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7006847-711c-497c-8a6c-04cdea8f653c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:21.592584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-b7006847-711c-497c-8a6c-04cdea8f653c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:34:21.592584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:21.592584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:21.592584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:21.592584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:21.592584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:21.592584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:21.592584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:21.592584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:21.592584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:21.592584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:21.592584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:21.592584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:34:21.592584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:34:21.592584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:34:21.592584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:21.593639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:21.594600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:21.594600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:21.594600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:34:21.594600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:34:21.594600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:34:21.594600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:34:21.594600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:34:21.594600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:34:21.594600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:34:21.594600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:21.598794 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 615/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:21.600390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331684ce-4b3a-4256-a6bd-877c79e5e3e8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:21.660466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:21.719494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331684ce-4b3a-4256-a6bd-877c79e5e3e8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:21.757542 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e9d36-d47c-4b26-83dd-f6b9e760eb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:21.758201 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e9d36-d47c-4b26-83dd-f6b9e760eb30 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:21.758473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e9d36-d47c-4b26-83dd-f6b9e760eb30 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:21.758779 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e9d36-d47c-4b26-83dd-f6b9e760eb30 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:21.759542 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e9d36-d47c-4b26-83dd-f6b9e760eb30 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:21.759878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e9d36-d47c-4b26-83dd-f6b9e760eb30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:21.761571 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e9d36-d47c-4b26-83dd-f6b9e760eb30 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:21.785033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 612/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Aug 30 14:34:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:21.790896 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7006847-711c-497c-8a6c-04cdea8f653c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:21.901976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7006847-711c-497c-8a6c-04cdea8f653c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:21.947534 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40acdced-d86f-4256-bbab-f38bae059f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:21.948432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acdced-d86f-4256-bbab-f38bae059f9b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:21.948959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acdced-d86f-4256-bbab-f38bae059f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:21.949292 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acdced-d86f-4256-bbab-f38bae059f9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:21.965356 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40acdced-d86f-4256-bbab-f38bae059f9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:21.967424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40acdced-d86f-4256-bbab-f38bae059f9b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:21.986058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-40acdced-d86f-4256-bbab-f38bae059f9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:21.986058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:21.986058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:21.986058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:21.986058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:21.986058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:21.986058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:21.986058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:21.986058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:21.986058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:21.986058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:21.986058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:21.986058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:34:21.986058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:34:21.986058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:34:21.986058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:34:21.988090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:21.988090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:21.988090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:21.988090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:21.988090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:21.988090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:21.988090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:21.988090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:21.988090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:21.988090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:21.988090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:21.988090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:21.988090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:21.988090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:21.988090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:21.988090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:21.988090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:34:21.989322 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:34:21.989322 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:21.989322 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:21.989322 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 616/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:21.994508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e9d36-d47c-4b26-83dd-f6b9e760eb30 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:22.106123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e9d36-d47c-4b26-83dd-f6b9e760eb30 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:22.108684 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:22.142363 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999e174-a14f-41a2-a81e-e27f2b166d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:22.142980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999e174-a14f-41a2-a81e-e27f2b166d64 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:22.143317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999e174-a14f-41a2-a81e-e27f2b166d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:22.143585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999e174-a14f-41a2-a81e-e27f2b166d64 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:22.144267 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999e174-a14f-41a2-a81e-e27f2b166d64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:22.144566 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999e174-a14f-41a2-a81e-e27f2b166d64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:22.145950 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999e174-a14f-41a2-a81e-e27f2b166d64 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:22.161037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 613/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:22.167130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40acdced-d86f-4256-bbab-f38bae059f9b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:22.177915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:22.274987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40acdced-d86f-4256-bbab-f38bae059f9b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:22.309755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74055949-8ba3-433a-adf0-984748f2cd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:22.310396 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74055949-8ba3-433a-adf0-984748f2cd09 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:22.310669 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74055949-8ba3-433a-adf0-984748f2cd09 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:22.310936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74055949-8ba3-433a-adf0-984748f2cd09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:22.317928 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74055949-8ba3-433a-adf0-984748f2cd09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:22.319364 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74055949-8ba3-433a-adf0-984748f2cd09 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:22.333052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-74055949-8ba3-433a-adf0-984748f2cd09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:34:22.333052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:22.333052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:22.333052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:22.333052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:22.333052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:22.333052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:22.333052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:22.333052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:22.333052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:22.333052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:22.333052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:22.333052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:34:22.333052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:34:22.333052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:34:22.334052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:34:22.334052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:22.334052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:22.334052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:22.334052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:22.334052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:22.334052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:22.334052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:22.334052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:22.334052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:22.334052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:22.334052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:22.334052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:22.334052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:22.334052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:22.334052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:22.334052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:22.340269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:34:22.340269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:34:22.340269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:34:22.340269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:34:22.340269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:34:22.340269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:34:22.340269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:34:22.340269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:22.340269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 617/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:34:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:22.341032 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999e174-a14f-41a2-a81e-e27f2b166d64 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:22.458741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999e174-a14f-41a2-a81e-e27f2b166d64 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:22.494097 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d5398-4d3a-498a-ab30-879bfa937cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:22.494720 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d5398-4d3a-498a-ab30-879bfa937cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:22.494997 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d5398-4d3a-498a-ab30-879bfa937cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:22.495265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d5398-4d3a-498a-ab30-879bfa937cd2 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:22.496113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d5398-4d3a-498a-ab30-879bfa937cd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:22.496418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d5398-4d3a-498a-ab30-879bfa937cd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:22.497855 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d5398-4d3a-498a-ab30-879bfa937cd2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:22.515666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 614/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:34:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:22.517160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74055949-8ba3-433a-adf0-984748f2cd09 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:22.595766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:22.637420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74055949-8ba3-433a-adf0-984748f2cd09 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:22.674768 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cb596-dc37-4032-9d25-d462c8c7d431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:22.675438 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cb596-dc37-4032-9d25-d462c8c7d431 None admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:22.675705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cb596-dc37-4032-9d25-d462c8c7d431 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:22.675976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cb596-dc37-4032-9d25-d462c8c7d431 None admin] PATH_INFO: `/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/d7cdd6b1ee9f4e97beb9bb037e446c8f/roles/f0fd911d7c9246aeb3d67299473a9f42` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:22.692335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514cb596-dc37-4032-9d25-d462c8c7d431 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9c57a581324217ad2e64a0c9c7fdee, user_id=d7cdd6b1ee9f4e97beb9bb037e446c8f, role_id=f0fd911d7c9246aeb3d67299473a9f42)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:22.693790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514cb596-dc37-4032-9d25-d462c8c7d431 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:22.712257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 618/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:34:22 2022] PUT /identity/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/d7cdd6b1ee9f4e97beb9bb037e446c8f/roles/f0fd911d7c9246aeb3d67299473a9f42 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:34:24.646406 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8fe62-4bd2-40ca-8c7a-44e758feb806 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:24.646981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8fe62-4bd2-40ca-8c7a-44e758feb806 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:24.647085 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8fe62-4bd2-40ca-8c7a-44e758feb806 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:24.648681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 615/1233] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:34:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:24.654411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14068e62-483c-4451-88bf-d67b49a202f1 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:24.654918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14068e62-483c-4451-88bf-d67b49a202f1 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:24.655004 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14068e62-483c-4451-88bf-d67b49a202f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:24.673878 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-14068e62-483c-4451-88bf-d67b49a202f1 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:24.675081 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-14068e62-483c-4451-88bf-d67b49a202f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:24.743574 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 619/1234] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:34:25.013864 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8fe62-4bd2-40ca-8c7a-44e758feb806 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:25.042284 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:25.131756 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8fe62-4bd2-40ca-8c7a-44e758feb806 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:25.170520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f36e9c-9929-417a-8ac7-e69056794abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:25.171161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f36e9c-9929-417a-8ac7-e69056794abf None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:25.171471 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f36e9c-9929-417a-8ac7-e69056794abf None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:25.171689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f36e9c-9929-417a-8ac7-e69056794abf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:25.182736 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f36e9c-9929-417a-8ac7-e69056794abf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:34:25.187709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-64f36e9c-9929-417a-8ac7-e69056794abf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:34:25.187709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:25.187709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:25.187709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:25.187709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:25.187709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:25.187709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:25.187709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:25.187709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:25.187709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:25.187709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:25.187709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:25.187709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:34:25.187709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:34:25.187709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:34:25.187709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:34:25.188870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:34:25.188870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:34:25.188870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:34:25.188870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:34:25.188870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:25.188870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:25.188870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:25.188870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:25.188870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:25.188870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:25.188870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:25.188870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:25.188870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:25.188870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:25.188870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:25.188870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:25.189806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:25.189806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:25.189806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:25.189806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:25.189806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:34:25.189806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:34:25.189806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:34:25.189806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:34:25.189806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:34:25.189806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:34:25.189806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:34:25.189806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:34:25.189806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 616/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Aug 30 14:34:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:25.197652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14068e62-483c-4451-88bf-d67b49a202f1 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:25.247764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:25.311954 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14068e62-483c-4451-88bf-d67b49a202f1 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:25.353803 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078d87c-2147-4943-894d-6f370a99d61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:25.354506 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078d87c-2147-4943-894d-6f370a99d61a None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:25.355045 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078d87c-2147-4943-894d-6f370a99d61a None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:25.355179 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078d87c-2147-4943-894d-6f370a99d61a None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:25.355919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078d87c-2147-4943-894d-6f370a99d61a None admin] RBAC: Adding query filter params (name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:25.356250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078d87c-2147-4943-894d-6f370a99d61a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:25.357973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078d87c-2147-4943-894d-6f370a99d61a None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:25.369187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 620/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Aug 30 14:34:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:25.376773 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f36e9c-9929-417a-8ac7-e69056794abf None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:25.483075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:25.506863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f36e9c-9929-417a-8ac7-e69056794abf None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:25.549039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82888b83-584d-4fcb-8bdd-666ca3543d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:25.549705 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82888b83-584d-4fcb-8bdd-666ca3543d98 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:25.549986 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82888b83-584d-4fcb-8bdd-666ca3543d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:25.550228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82888b83-584d-4fcb-8bdd-666ca3543d98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:25.564443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82888b83-584d-4fcb-8bdd-666ca3543d98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:25.565868 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82888b83-584d-4fcb-8bdd-666ca3543d98 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:25.583071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-82888b83-584d-4fcb-8bdd-666ca3543d98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:25.583071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:25.583071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:25.583071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:25.583071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:25.583071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:25.583071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:25.583071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:25.583071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:25.583071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:25.583071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:25.583071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:25.583071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:34:25.583071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:34:25.583071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:34:25.583071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:34:25.584231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:25.584231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:25.584231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:25.584231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:25.584231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:25.584231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:25.584231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:25.584231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:25.584231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:25.584231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:25.584231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:25.584231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:25.584231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:25.584231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:25.584231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:25.584231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:25.584231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:34:25.585132 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:34:25.585132 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:25.585132 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:34:25.585132 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 617/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:34:25.590835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078d87c-2147-4943-894d-6f370a99d61a None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:25.708433 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078d87c-2147-4943-894d-6f370a99d61a None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:25.718994 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:25.748220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f8ea1-13e9-44dc-be75-118e1a76201b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:25.748924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f8ea1-13e9-44dc-be75-118e1a76201b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:25.749265 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f8ea1-13e9-44dc-be75-118e1a76201b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:25.749523 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f8ea1-13e9-44dc-be75-118e1a76201b None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:25.750238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f8ea1-13e9-44dc-be75-118e1a76201b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:25.750535 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f8ea1-13e9-44dc-be75-118e1a76201b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:25.751917 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f8ea1-13e9-44dc-be75-118e1a76201b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:25.765719 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 621/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:34:25.771900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82888b83-584d-4fcb-8bdd-666ca3543d98 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:25.893657 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82888b83-584d-4fcb-8bdd-666ca3543d98 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:25.941278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7f319-3a59-46a8-b301-70b8ef42e6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:25.942434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7f319-3a59-46a8-b301-70b8ef42e6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:25.942779 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7f319-3a59-46a8-b301-70b8ef42e6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:25.943006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7f319-3a59-46a8-b301-70b8ef42e6d0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:25.947055 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:25.955579 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7f319-3a59-46a8-b301-70b8ef42e6d0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:25.957334 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7f319-3a59-46a8-b301-70b8ef42e6d0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:25.974691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-cbd7f319-3a59-46a8-b301-70b8ef42e6d0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:34:25.974691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:25.974691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:25.974691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:25.974691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:25.974691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:25.974691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:25.974691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:25.974691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:25.974691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:25.974691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:25.974691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:25.974691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:34:25.974691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:34:25.974691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:34:25.974691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:25.977134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:25.978765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:25.978765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:25.978765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:34:25.978765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:34:25.978765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:34:25.978765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:34:25.978765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:34:25.978765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:34:25.978765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:34:25.978765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:34:25.978765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 618/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:34:25.983707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f8ea1-13e9-44dc-be75-118e1a76201b None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:26.100296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f8ea1-13e9-44dc-be75-118e1a76201b None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:26.145923 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e25dcb-d5b7-4909-a175-468b121d663c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:26.146834 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e25dcb-d5b7-4909-a175-468b121d663c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:26.147283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e25dcb-d5b7-4909-a175-468b121d663c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:26.147729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e25dcb-d5b7-4909-a175-468b121d663c None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:26.148782 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e25dcb-d5b7-4909-a175-468b121d663c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:26.149257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e25dcb-d5b7-4909-a175-468b121d663c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:26.150951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e25dcb-d5b7-4909-a175-468b121d663c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:26.180478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 622/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Aug 30 14:34:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:34:26.187349 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7f319-3a59-46a8-b301-70b8ef42e6d0 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:26.316753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7f319-3a59-46a8-b301-70b8ef42e6d0 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:26.355921 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380fa51-b48b-4d77-90b4-d6fe25905eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:26.356583 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380fa51-b48b-4d77-90b4-d6fe25905eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:26.356856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380fa51-b48b-4d77-90b4-d6fe25905eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:26.357120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380fa51-b48b-4d77-90b4-d6fe25905eb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:26.427383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:26.435281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380fa51-b48b-4d77-90b4-d6fe25905eb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:26.437071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380fa51-b48b-4d77-90b4-d6fe25905eb8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:26.550491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 619/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:34:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:34:26.556386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e25dcb-d5b7-4909-a175-468b121d663c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:26.587235 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:26.663452 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e25dcb-d5b7-4909-a175-468b121d663c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:26.700098 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a978c80-ea52-4bd8-b635-82b33f8b01bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:26.700696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a978c80-ea52-4bd8-b635-82b33f8b01bf None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:26.700987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a978c80-ea52-4bd8-b635-82b33f8b01bf None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:26.701239 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a978c80-ea52-4bd8-b635-82b33f8b01bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:26.712733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a978c80-ea52-4bd8-b635-82b33f8b01bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:26.714435 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a978c80-ea52-4bd8-b635-82b33f8b01bf None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:26.730763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-0a978c80-ea52-4bd8-b635-82b33f8b01bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:26.730763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:26.730763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:26.730763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:26.730763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:26.730763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:26.730763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:26.730763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:26.730763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:26.730763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:26.730763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:26.730763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:26.730763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:34:26.730763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:34:26.730763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:34:26.730763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:34:26.731982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:26.731982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:26.731982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:26.731982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:26.731982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:26.731982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:26.731982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:26.731982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:26.731982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:26.731982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:26.731982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:26.731982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:26.731982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:26.731982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:26.731982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:26.731982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:26.731982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:34:26.733095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:34:26.733095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:26.733095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:26.733095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 623/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:26.737989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380fa51-b48b-4d77-90b4-d6fe25905eb8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:26.843658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:26.850187 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380fa51-b48b-4d77-90b4-d6fe25905eb8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:26.889930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4751d9-04bb-4864-95b9-16c9aeb0c211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:26.890793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4751d9-04bb-4864-95b9-16c9aeb0c211 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:26.891070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4751d9-04bb-4864-95b9-16c9aeb0c211 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:26.891335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4751d9-04bb-4864-95b9-16c9aeb0c211 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:26.891987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4751d9-04bb-4864-95b9-16c9aeb0c211 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:26.892311 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4751d9-04bb-4864-95b9-16c9aeb0c211 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:26.893691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4751d9-04bb-4864-95b9-16c9aeb0c211 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:26.905708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 620/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:34:26.911606 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a978c80-ea52-4bd8-b635-82b33f8b01bf None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:27.019429 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:27.044964 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a978c80-ea52-4bd8-b635-82b33f8b01bf None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:27.081427 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db50a2-104e-455f-a91c-671c42637b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:27.082068 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db50a2-104e-455f-a91c-671c42637b12 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:27.082346 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db50a2-104e-455f-a91c-671c42637b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:27.082618 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db50a2-104e-455f-a91c-671c42637b12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:27.089592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db50a2-104e-455f-a91c-671c42637b12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:27.091062 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db50a2-104e-455f-a91c-671c42637b12 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:27.102847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-90db50a2-104e-455f-a91c-671c42637b12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:34:27.102847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:27.102847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:27.102847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:27.102847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:27.102847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:27.102847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:27.102847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:27.102847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:27.102847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:27.102847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:27.102847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:27.102847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:34:27.102847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:34:27.102847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:34:27.104397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:34:27.104397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:27.104397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:27.104397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:27.104397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:27.104397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:27.104397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:27.104397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:27.104397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:27.104397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:27.104397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:27.104397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:27.104397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:27.104397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:27.104397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:27.104397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:27.104397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:27.106018 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:34:27.106018 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:34:27.106018 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:34:27.106018 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:34:27.106018 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:34:27.106018 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:34:27.106018 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:34:27.106018 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:27.106018 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 624/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:34:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:27.110243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4751d9-04bb-4864-95b9-16c9aeb0c211 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:27.260387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4751d9-04bb-4864-95b9-16c9aeb0c211 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:27.303681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd54b7-0f6b-4d00-8a1c-2850a80a2271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:27.304250 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd54b7-0f6b-4d00-8a1c-2850a80a2271 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:27.304560 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd54b7-0f6b-4d00-8a1c-2850a80a2271 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:27.304825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd54b7-0f6b-4d00-8a1c-2850a80a2271 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:27.305592 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd54b7-0f6b-4d00-8a1c-2850a80a2271 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:27.305936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd54b7-0f6b-4d00-8a1c-2850a80a2271 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:27.307460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd54b7-0f6b-4d00-8a1c-2850a80a2271 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:27.324608 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 621/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:34:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:27.331830 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db50a2-104e-455f-a91c-671c42637b12 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:27.481427 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db50a2-104e-455f-a91c-671c42637b12 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:27.525197 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0272a-6bd6-4823-a27a-f17125b22e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:27.525819 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0272a-6bd6-4823-a27a-f17125b22e0d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:27.526100 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0272a-6bd6-4823-a27a-f17125b22e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:27.526375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0272a-6bd6-4823-a27a-f17125b22e0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:27.528022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0272a-6bd6-4823-a27a-f17125b22e0d None admin] RBAC: Adding query filter params (role.id=f0fd911d7c9246aeb3d67299473a9f42, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=d7cdd6b1ee9f4e97beb9bb037e446c8f) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:27.528341 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0272a-6bd6-4823-a27a-f17125b22e0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:27.529817 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0272a-6bd6-4823-a27a-f17125b22e0d None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:27.545050 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 625/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:34:27 2022] GET /identity/v3/role_assignments?user.id=d7cdd6b1ee9f4e97beb9bb037e446c8f&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=f0fd911d7c9246aeb3d67299473a9f42 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:29.527779 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f525b-21a2-44a2-854b-815d1c07e365 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:29.528484 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f525b-21a2-44a2-854b-815d1c07e365 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:29.528484 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f525b-21a2-44a2-854b-815d1c07e365 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:29.529779 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 622/1247] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:29.536407 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d1414-810a-4984-9e38-3cf55082dcce None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:29.537202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d1414-810a-4984-9e38-3cf55082dcce None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:29.537399 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d1414-810a-4984-9e38-3cf55082dcce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:29.545822 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:29.562814 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-4e3d1414-810a-4984-9e38-3cf55082dcce None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:29.564129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4e3d1414-810a-4984-9e38-3cf55082dcce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:29.644615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 626/1248] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:34:29.933371 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f525b-21a2-44a2-854b-815d1c07e365 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:29.957279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:30.079992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f525b-21a2-44a2-854b-815d1c07e365 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:30.120284 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1632f-8f1e-4317-99b5-e2a78e8c9b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:30.121260 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1632f-8f1e-4317-99b5-e2a78e8c9b5e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:30.121260 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1632f-8f1e-4317-99b5-e2a78e8c9b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:30.121498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1632f-8f1e-4317-99b5-e2a78e8c9b5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:30.134548 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1632f-8f1e-4317-99b5-e2a78e8c9b5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:34:30.140025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-53e1632f-8f1e-4317-99b5-e2a78e8c9b5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:34:30.140025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:30.140025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:30.140025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:30.140025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:30.140025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:30.140025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:30.140025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:30.140025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:30.140025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:30.140025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:30.140025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:30.140025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:34:30.140025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:34:30.140025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:34:30.140025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:34:30.141231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:34:30.141231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:34:30.141231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:34:30.141231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:34:30.141231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:30.141231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:30.141231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:30.141231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:30.141231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:30.141231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:30.141231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:30.141231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:30.141231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:30.141231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:30.141231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:30.141231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:30.142164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:30.142164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:30.142164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:30.142164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:30.142164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:34:30.142164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:34:30.142164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:34:30.142164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:34:30.142164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:34:30.142164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:34:30.142164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:34:30.142164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:34:30.142164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 623/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:34:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:30.148779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d1414-810a-4984-9e38-3cf55082dcce None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:30.275403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d1414-810a-4984-9e38-3cf55082dcce None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:30.313785 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:30.320568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dfbd02-8cb3-4dba-8c0b-e27ff2ce26a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:30.321250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfbd02-8cb3-4dba-8c0b-e27ff2ce26a7 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:30.321632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfbd02-8cb3-4dba-8c0b-e27ff2ce26a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:30.321997 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfbd02-8cb3-4dba-8c0b-e27ff2ce26a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:30.322940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dfbd02-8cb3-4dba-8c0b-e27ff2ce26a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:30.323764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dfbd02-8cb3-4dba-8c0b-e27ff2ce26a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:30.325996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dfbd02-8cb3-4dba-8c0b-e27ff2ce26a7 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:30.337980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 627/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:34:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:30.345725 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1632f-8f1e-4317-99b5-e2a78e8c9b5e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:30.443168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:30.473366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1632f-8f1e-4317-99b5-e2a78e8c9b5e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:30.515188 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1bcb6-9999-477c-a0c6-77bc9fe68e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:30.515848 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1bcb6-9999-477c-a0c6-77bc9fe68e57 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:30.516151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1bcb6-9999-477c-a0c6-77bc9fe68e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:30.516418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1bcb6-9999-477c-a0c6-77bc9fe68e57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:30.530003 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1bcb6-9999-477c-a0c6-77bc9fe68e57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:30.531374 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1bcb6-9999-477c-a0c6-77bc9fe68e57 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:30.548551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-34c1bcb6-9999-477c-a0c6-77bc9fe68e57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:30.548551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:30.548551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:30.548551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:30.548551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:30.548551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:30.548551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:30.548551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:30.548551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:30.548551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:30.548551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:30.548551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:30.548551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:34:30.548551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:34:30.548551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:34:30.548551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:34:30.550404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:30.550404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:30.550404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:30.550404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:30.550404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:30.550404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:30.550404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:30.550404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:30.550404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:30.550404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:30.550404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:30.550404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:30.550404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:30.550404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:30.550404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:30.550404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:30.550404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:34:30.551451 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:34:30.551451 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:30.551451 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:34:30.551451 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 624/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:30.561337 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dfbd02-8cb3-4dba-8c0b-e27ff2ce26a7 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:30.706062 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dfbd02-8cb3-4dba-8c0b-e27ff2ce26a7 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:30.758672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c77d8a-2ee7-43eb-8030-51b9183b06cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:30.759424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c77d8a-2ee7-43eb-8030-51b9183b06cd None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:30.759770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c77d8a-2ee7-43eb-8030-51b9183b06cd None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:30.760225 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c77d8a-2ee7-43eb-8030-51b9183b06cd None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:30.761129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c77d8a-2ee7-43eb-8030-51b9183b06cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:30.761542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c77d8a-2ee7-43eb-8030-51b9183b06cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:30.763177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c77d8a-2ee7-43eb-8030-51b9183b06cd None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:30.782618 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 628/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:30.789394 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1bcb6-9999-477c-a0c6-77bc9fe68e57 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:30.914069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1bcb6-9999-477c-a0c6-77bc9fe68e57 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:30.966716 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:30.980267 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9204a8-2ba8-4af7-be97-e0566f4b1e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:30.981027 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9204a8-2ba8-4af7-be97-e0566f4b1e09 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:30.981411 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9204a8-2ba8-4af7-be97-e0566f4b1e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:30.981766 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9204a8-2ba8-4af7-be97-e0566f4b1e09 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:30.993425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9204a8-2ba8-4af7-be97-e0566f4b1e09 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:30.995384 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9204a8-2ba8-4af7-be97-e0566f4b1e09 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:31.013807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-1f9204a8-2ba8-4af7-be97-e0566f4b1e09 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:34:31.013807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:31.013807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:31.013807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:31.013807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:31.013807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:31.013807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:31.013807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:31.013807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:31.013807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:31.013807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:31.013807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:31.013807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:34:31.013807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:34:31.013807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:34:31.013807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:31.015031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:31.016182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:31.016182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:31.016182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:34:31.016182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:34:31.016182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:34:31.016182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:34:31.016182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:34:31.016182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:34:31.016182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:34:31.016182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:34:31.016182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 625/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:31.021543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c77d8a-2ee7-43eb-8030-51b9183b06cd None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:31.101664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:31.159511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c77d8a-2ee7-43eb-8030-51b9183b06cd None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:31.203302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d249e-ee98-4292-ada2-6495fb7e37a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:31.203964 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d249e-ee98-4292-ada2-6495fb7e37a4 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:31.204643 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d249e-ee98-4292-ada2-6495fb7e37a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:31.204934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d249e-ee98-4292-ada2-6495fb7e37a4 None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:31.205685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d249e-ee98-4292-ada2-6495fb7e37a4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:31.205983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d249e-ee98-4292-ada2-6495fb7e37a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:31.207422 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d249e-ee98-4292-ada2-6495fb7e37a4 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:31.227922 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 629/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Aug 30 14:34:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:31.235678 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9204a8-2ba8-4af7-be97-e0566f4b1e09 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:31.364057 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9204a8-2ba8-4af7-be97-e0566f4b1e09 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:31.404275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274f0b6-c356-4420-ae63-a31edbee5d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:31.404915 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274f0b6-c356-4420-ae63-a31edbee5d06 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:31.405217 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274f0b6-c356-4420-ae63-a31edbee5d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:31.405456 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274f0b6-c356-4420-ae63-a31edbee5d06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:31.486253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:31.504674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274f0b6-c356-4420-ae63-a31edbee5d06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:31.506204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274f0b6-c356-4420-ae63-a31edbee5d06 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:31.659157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 626/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:34:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:34:31.665396 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d249e-ee98-4292-ada2-6495fb7e37a4 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:31.802314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d249e-ee98-4292-ada2-6495fb7e37a4 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:31.819453 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:31.845168 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a0d59-0271-4f89-943b-95871ef67b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:31.846082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a0d59-0271-4f89-943b-95871ef67b60 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:31.846942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a0d59-0271-4f89-943b-95871ef67b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:31.847251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a0d59-0271-4f89-943b-95871ef67b60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:31.861828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a0d59-0271-4f89-943b-95871ef67b60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:31.864145 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a0d59-0271-4f89-943b-95871ef67b60 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:31.888125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-df4a0d59-0271-4f89-943b-95871ef67b60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:31.888125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:31.888125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:31.888125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:31.888125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:31.888125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:31.888125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:31.888125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:31.888125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:31.888125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:31.888125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:31.888125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:31.888125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:34:31.888125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:34:31.888125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:34:31.888125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:34:31.890271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:31.890271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:31.890271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:31.890271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:31.890271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:31.890271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:31.890271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:31.890271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:31.890271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:31.890271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:31.890271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:31.890271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:31.890271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:31.890271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:31.890271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:31.890271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:31.890271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:34:31.892040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:34:31.892040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:31.892040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:31.892040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 630/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:31.896360 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274f0b6-c356-4420-ae63-a31edbee5d06 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:32.013224 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:32.028338 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274f0b6-c356-4420-ae63-a31edbee5d06 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:32.067155 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861648f4-902e-4c8f-b8b1-b05cd68761b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:32.067862 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861648f4-902e-4c8f-b8b1-b05cd68761b4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:32.068141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861648f4-902e-4c8f-b8b1-b05cd68761b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:32.068355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861648f4-902e-4c8f-b8b1-b05cd68761b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:32.069086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861648f4-902e-4c8f-b8b1-b05cd68761b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:32.069395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861648f4-902e-4c8f-b8b1-b05cd68761b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:32.070828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861648f4-902e-4c8f-b8b1-b05cd68761b4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:32.083608 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 627/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:32.092397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a0d59-0271-4f89-943b-95871ef67b60 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:32.203940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a0d59-0271-4f89-943b-95871ef67b60 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:32.243266 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20a011-37c9-4c65-8e7c-fcd8219bbf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:32.244190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20a011-37c9-4c65-8e7c-fcd8219bbf98 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:32.244638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20a011-37c9-4c65-8e7c-fcd8219bbf98 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:32.245191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20a011-37c9-4c65-8e7c-fcd8219bbf98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:32.255067 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20a011-37c9-4c65-8e7c-fcd8219bbf98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:32.257112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20a011-37c9-4c65-8e7c-fcd8219bbf98 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:32.273860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-fe20a011-37c9-4c65-8e7c-fcd8219bbf98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:34:32.273860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:32.273860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:32.273860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:32.273860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:32.273860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:32.273860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:32.273860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:32.273860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:32.273860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:32.273860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:32.273860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:32.273860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:34:32.273860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:34:32.273860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:34:32.275807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:34:32.275807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:32.275807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:32.275807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:32.275807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:32.275807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:32.275807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:32.275807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:32.275807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:32.275807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:32.275807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:32.275807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:32.275807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:32.275807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:32.275807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:32.275807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:32.275807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:32.283379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:34:32.283379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:34:32.283379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:34:32.283379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:34:32.283379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:34:32.283379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:34:32.283379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:34:32.283379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:32.283379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 631/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:34:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:32.284400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861648f4-902e-4c8f-b8b1-b05cd68761b4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:32.408987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861648f4-902e-4c8f-b8b1-b05cd68761b4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:32.451140 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed37154-e006-4136-9254-f683e6ca92ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:32.451774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed37154-e006-4136-9254-f683e6ca92ad None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:32.452086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed37154-e006-4136-9254-f683e6ca92ad None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:32.452366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed37154-e006-4136-9254-f683e6ca92ad None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:32.453049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed37154-e006-4136-9254-f683e6ca92ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:32.453357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed37154-e006-4136-9254-f683e6ca92ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:32.455203 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed37154-e006-4136-9254-f683e6ca92ad None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:32.470275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 628/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:34:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:32.477075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20a011-37c9-4c65-8e7c-fcd8219bbf98 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:32.508785 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:32.601620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20a011-37c9-4c65-8e7c-fcd8219bbf98 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:32.647359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80c508-c00b-4057-b2d2-619bc39db22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:32.648414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80c508-c00b-4057-b2d2-619bc39db22b None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:32.648873 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80c508-c00b-4057-b2d2-619bc39db22b None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:32.649336 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80c508-c00b-4057-b2d2-619bc39db22b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:32.650649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80c508-c00b-4057-b2d2-619bc39db22b None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=d7cdd6b1ee9f4e97beb9bb037e446c8f) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:32.651164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80c508-c00b-4057-b2d2-619bc39db22b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:32.654083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80c508-c00b-4057-b2d2-619bc39db22b None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:32.669025 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 632/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:34:32 2022] GET /identity/v3/role_assignments?user.id=d7cdd6b1ee9f4e97beb9bb037e446c8f&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:34.670436 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85519a-5d6d-4f21-b3d3-45e982be18a6 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:34.671006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85519a-5d6d-4f21-b3d3-45e982be18a6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:34.671006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85519a-5d6d-4f21-b3d3-45e982be18a6 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:34.672314 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 629/1261] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:34.677923 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859689d0-724b-4686-be94-249275d93dd1 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:34.678386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859689d0-724b-4686-be94-249275d93dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:34.678522 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859689d0-724b-4686-be94-249275d93dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:34.699222 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-859689d0-724b-4686-be94-249275d93dd1 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:34.700648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-859689d0-724b-4686-be94-249275d93dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:34.774731 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 633/1262] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:34:35.057833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85519a-5d6d-4f21-b3d3-45e982be18a6 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:35.069373 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:35.170728 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85519a-5d6d-4f21-b3d3-45e982be18a6 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:35.206695 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4c30a-161a-4f19-b513-51497d17d50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:35.207339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4c30a-161a-4f19-b513-51497d17d50e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:35.207613 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4c30a-161a-4f19-b513-51497d17d50e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:35.207877 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4c30a-161a-4f19-b513-51497d17d50e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:35.218878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4c30a-161a-4f19-b513-51497d17d50e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:34:35.223744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-0ee4c30a-161a-4f19-b513-51497d17d50e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:34:35.223744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:35.223744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:35.223744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:35.223744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:35.223744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:35.223744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:35.223744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:35.223744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:35.223744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:35.223744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:35.223744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:35.223744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:34:35.223744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:34:35.223744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:34:35.223744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:34:35.224903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:34:35.224903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:34:35.224903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:34:35.224903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:34:35.224903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:35.224903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:35.224903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:35.224903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:35.224903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:35.224903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:35.224903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:35.224903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:35.224903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:35.224903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:35.224903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:35.224903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:35.226237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:35.226237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:35.226237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:35.226237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:35.226237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:34:35.226237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:34:35.226237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:34:35.226237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:34:35.226237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:34:35.226237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:34:35.226237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:34:35.226237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:34:35.226237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 630/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:34:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:35.231393 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859689d0-724b-4686-be94-249275d93dd1 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:35.271689 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:35.392638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859689d0-724b-4686-be94-249275d93dd1 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:35.436359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62147944-4876-457b-9daa-1f177b9ea827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:35.437070 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62147944-4876-457b-9daa-1f177b9ea827 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:35.437375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62147944-4876-457b-9daa-1f177b9ea827 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:35.437642 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62147944-4876-457b-9daa-1f177b9ea827 None admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:35.438342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62147944-4876-457b-9daa-1f177b9ea827 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:35.438713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62147944-4876-457b-9daa-1f177b9ea827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:35.440170 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62147944-4876-457b-9daa-1f177b9ea827 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:35.450596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 634/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:34:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:35.459719 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4c30a-161a-4f19-b513-51497d17d50e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:35.549837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:35.586872 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4c30a-161a-4f19-b513-51497d17d50e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:35.623521 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2505fb7-f6fc-4def-9b5f-33598f632ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:35.624280 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2505fb7-f6fc-4def-9b5f-33598f632ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:35.624679 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2505fb7-f6fc-4def-9b5f-33598f632ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:35.624950 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2505fb7-f6fc-4def-9b5f-33598f632ee0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:35.636653 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2505fb7-f6fc-4def-9b5f-33598f632ee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:35.638201 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2505fb7-f6fc-4def-9b5f-33598f632ee0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:35.654492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-d2505fb7-f6fc-4def-9b5f-33598f632ee0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:35.654492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:35.654492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:35.654492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:35.654492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:35.654492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:35.654492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:35.654492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:35.654492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:35.654492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:35.654492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:35.654492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:35.654492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:34:35.654492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:34:35.654492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:34:35.654492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:34:35.655789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:35.655789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:35.655789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:35.655789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:35.655789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:35.655789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:35.655789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:35.655789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:35.655789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:35.655789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:35.655789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:35.655789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:35.655789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:35.655789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:35.655789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:35.655789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:35.655789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:34:35.656881 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:34:35.656881 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:35.656881 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:34:35.656881 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 631/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:35.661912 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62147944-4876-457b-9daa-1f177b9ea827 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:35.769077 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:35.795196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62147944-4876-457b-9daa-1f177b9ea827 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:35.838020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f9105-9c05-43cf-8522-592c7443ad9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:35.838653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f9105-9c05-43cf-8522-592c7443ad9d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:35.839005 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f9105-9c05-43cf-8522-592c7443ad9d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:35.839306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f9105-9c05-43cf-8522-592c7443ad9d None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:35.839966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f9105-9c05-43cf-8522-592c7443ad9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:35.840330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f9105-9c05-43cf-8522-592c7443ad9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:35.841720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f9105-9c05-43cf-8522-592c7443ad9d None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:35.855481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 635/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:35.861527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2505fb7-f6fc-4def-9b5f-33598f632ee0 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:35.982196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2505fb7-f6fc-4def-9b5f-33598f632ee0 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:36.008860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:36.025554 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894f9d8-83ec-48a4-b524-072ee00c9e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:36.026207 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894f9d8-83ec-48a4-b524-072ee00c9e5b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:36.026531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894f9d8-83ec-48a4-b524-072ee00c9e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:36.026789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894f9d8-83ec-48a4-b524-072ee00c9e5b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:36.035259 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4894f9d8-83ec-48a4-b524-072ee00c9e5b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:36.036749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4894f9d8-83ec-48a4-b524-072ee00c9e5b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:36.049060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-4894f9d8-83ec-48a4-b524-072ee00c9e5b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:34:36.049060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:36.049060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:36.049060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:36.049060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:36.049060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:36.049060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:36.049060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:36.049060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:36.049060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:36.049060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:36.049060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:36.049060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:34:36.049060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:34:36.049060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:34:36.049060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:36.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:36.051209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:36.051209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:36.051209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:34:36.051209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:34:36.051209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:34:36.051209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:34:36.051209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:34:36.051209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:34:36.051209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:34:36.051209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:34:36.051209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 632/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:36.056660 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f9105-9c05-43cf-8522-592c7443ad9d None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:36.168635 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f9105-9c05-43cf-8522-592c7443ad9d None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:36.207953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8beb6a-aca9-48cc-a573-c879ada674ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:36.208660 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8beb6a-aca9-48cc-a573-c879ada674ce None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:36.208937 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8beb6a-aca9-48cc-a573-c879ada674ce None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:36.209200 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8beb6a-aca9-48cc-a573-c879ada674ce None admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:36.209959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8beb6a-aca9-48cc-a573-c879ada674ce None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:36.210260 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8beb6a-aca9-48cc-a573-c879ada674ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:36.211805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8beb6a-aca9-48cc-a573-c879ada674ce None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:36.223015 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:36.233164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 636/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Aug 30 14:34:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:36.242227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894f9d8-83ec-48a4-b524-072ee00c9e5b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:36.382729 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894f9d8-83ec-48a4-b524-072ee00c9e5b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:36.420481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824e298-bee0-4cdf-be1d-173018ab7119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:36.421163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824e298-bee0-4cdf-be1d-173018ab7119 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:36.421501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824e298-bee0-4cdf-be1d-173018ab7119 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:36.421774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824e298-bee0-4cdf-be1d-173018ab7119 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:36.434839 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824e298-bee0-4cdf-be1d-173018ab7119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:36.439707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824e298-bee0-4cdf-be1d-173018ab7119 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:36.459816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-3824e298-bee0-4cdf-be1d-173018ab7119 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:36.459816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:36.459816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:36.459816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:36.459816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:36.459816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:36.459816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:36.459816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:36.459816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:36.459816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:36.459816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:36.459816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:36.459816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:34:36.459816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:34:36.459816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:34:36.459816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:34:36.461293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:36.461293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:36.461293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:36.461293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:36.461293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:36.461293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:36.461293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:36.461293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:36.461293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:36.461293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:36.461293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:36.461293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:36.461293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:36.461293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:36.461293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:36.461293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:36.461293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:34:36.462366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:34:36.462366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:36.462366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:34:36.462366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 633/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:36.468794 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8beb6a-aca9-48cc-a573-c879ada674ce None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:36.586963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8beb6a-aca9-48cc-a573-c879ada674ce None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:36.633259 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baed03d-fbe7-4f31-8d0d-5ec42bc60d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:36.633885 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baed03d-fbe7-4f31-8d0d-5ec42bc60d06 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:36.634294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baed03d-fbe7-4f31-8d0d-5ec42bc60d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:36.634678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baed03d-fbe7-4f31-8d0d-5ec42bc60d06 None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:36.635422 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baed03d-fbe7-4f31-8d0d-5ec42bc60d06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:36.635726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baed03d-fbe7-4f31-8d0d-5ec42bc60d06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:36.637544 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baed03d-fbe7-4f31-8d0d-5ec42bc60d06 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:36.654770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 637/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:36.661522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824e298-bee0-4cdf-be1d-173018ab7119 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:36.699485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:36.778964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824e298-bee0-4cdf-be1d-173018ab7119 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:36.821271 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6e0ad-96ec-48c3-85f0-526bbdda3171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:36.821975 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6e0ad-96ec-48c3-85f0-526bbdda3171 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:36.822281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6e0ad-96ec-48c3-85f0-526bbdda3171 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:36.822609 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6e0ad-96ec-48c3-85f0-526bbdda3171 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:36.834150 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6e0ad-96ec-48c3-85f0-526bbdda3171 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:36.836264 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6e0ad-96ec-48c3-85f0-526bbdda3171 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:36.855697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-85e6e0ad-96ec-48c3-85f0-526bbdda3171 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:34:36.855697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:36.855697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:36.855697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:36.855697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:36.855697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:36.855697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:36.855697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:36.855697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:36.855697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:36.855697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:36.855697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:36.855697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:34:36.855697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:34:36.855697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:34:36.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:34:36.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:36.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:36.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:36.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:36.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:36.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:36.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:36.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:36.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:36.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:36.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:36.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:36.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:36.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:36.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:36.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:36.858335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:34:36.858335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:34:36.858335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:34:36.858335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:34:36.858335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:34:36.858335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:34:36.858335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:34:36.858335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:34:36.858335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 634/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:34:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:36.864298 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baed03d-fbe7-4f31-8d0d-5ec42bc60d06 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:36.952207 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:37.022620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baed03d-fbe7-4f31-8d0d-5ec42bc60d06 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:37.063496 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6650539-8868-4938-b198-20c5e46a236d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:37.064255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6650539-8868-4938-b198-20c5e46a236d None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:37.064531 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6650539-8868-4938-b198-20c5e46a236d None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:37.064800 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6650539-8868-4938-b198-20c5e46a236d None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:37.065499 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6650539-8868-4938-b198-20c5e46a236d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:37.065797 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6650539-8868-4938-b198-20c5e46a236d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:37.067227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6650539-8868-4938-b198-20c5e46a236d None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:37.081433 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 638/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:34:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:37.088230 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6e0ad-96ec-48c3-85f0-526bbdda3171 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:37.224701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:37.234868 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6e0ad-96ec-48c3-85f0-526bbdda3171 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:37.270357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe91568-3254-4789-ad84-e04269275327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:37.271025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe91568-3254-4789-ad84-e04269275327 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:37.271299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe91568-3254-4789-ad84-e04269275327 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:37.271628 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe91568-3254-4789-ad84-e04269275327 None admin] PATH_INFO: `/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/d7cdd6b1ee9f4e97beb9bb037e446c8f/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:37.273596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe91568-3254-4789-ad84-e04269275327 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9c57a581324217ad2e64a0c9c7fdee, user_id=d7cdd6b1ee9f4e97beb9bb037e446c8f, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:37.275107 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe91568-3254-4789-ad84-e04269275327 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:37.291356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 635/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Aug 30 14:34:37 2022] PUT /identity/v3/projects/1a9c57a581324217ad2e64a0c9c7fdee/users/d7cdd6b1ee9f4e97beb9bb037e446c8f/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:34:39.250838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce658b27-39f4-495d-844f-ccb304176434 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:39.251431 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce658b27-39f4-495d-844f-ccb304176434 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:39.251431 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce658b27-39f4-495d-844f-ccb304176434 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:39.252844 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 639/1274] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:39.258871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddc43e-babd-4086-b74b-1cbb494ec226 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:39.259425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddc43e-babd-4086-b74b-1cbb494ec226 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:39.259425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddc43e-babd-4086-b74b-1cbb494ec226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:39.280976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-44ddc43e-babd-4086-b74b-1cbb494ec226 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:39.282272 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-44ddc43e-babd-4086-b74b-1cbb494ec226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:39.358275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 636/1275] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:34:39.668271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce658b27-39f4-495d-844f-ccb304176434 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:39.791712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce658b27-39f4-495d-844f-ccb304176434 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:39.812510 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:39.832408 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec48730-7c17-44c2-86af-0dff58adecaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:39.833075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec48730-7c17-44c2-86af-0dff58adecaf None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:39.833344 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec48730-7c17-44c2-86af-0dff58adecaf None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:39.833682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec48730-7c17-44c2-86af-0dff58adecaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:39.845623 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec48730-7c17-44c2-86af-0dff58adecaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:34:39.850892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-cec48730-7c17-44c2-86af-0dff58adecaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:34:39.850892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:39.850892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:39.850892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:39.850892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:39.850892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:39.850892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:39.850892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:39.850892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:39.850892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:39.850892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:39.850892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:39.850892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:34:39.850892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:34:39.850892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:34:39.850892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:34:39.852538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:34:39.852538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:34:39.852538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:34:39.852538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:34:39.852538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:39.852538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:39.852538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:39.852538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:39.852538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:39.852538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:39.852538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:39.852538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:39.852538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:39.852538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:39.852538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:39.852538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:39.854283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:39.854283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:39.854283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:39.854283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:39.854283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:34:39.854283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:34:39.854283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:34:39.854283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:34:39.854283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:34:39.854283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:34:39.854283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:34:39.854283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:39.854283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 640/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Aug 30 14:34:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:39.860536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddc43e-babd-4086-b74b-1cbb494ec226 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:39.976301 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:40.007893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddc43e-babd-4086-b74b-1cbb494ec226 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:40.054087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de60c7-75ff-4a3b-a8ee-0f0c8e7b4624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:40.054833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de60c7-75ff-4a3b-a8ee-0f0c8e7b4624 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:40.055105 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de60c7-75ff-4a3b-a8ee-0f0c8e7b4624 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:40.055393 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de60c7-75ff-4a3b-a8ee-0f0c8e7b4624 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:40.056214 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de60c7-75ff-4a3b-a8ee-0f0c8e7b4624 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:40.056535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de60c7-75ff-4a3b-a8ee-0f0c8e7b4624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:40.058106 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de60c7-75ff-4a3b-a8ee-0f0c8e7b4624 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:40.068681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 637/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Aug 30 14:34:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:40.076274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec48730-7c17-44c2-86af-0dff58adecaf None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:40.197217 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec48730-7c17-44c2-86af-0dff58adecaf None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:40.234027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d221175-5689-4fdd-94b7-0d48fa4a520c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:40.234552 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d221175-5689-4fdd-94b7-0d48fa4a520c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:40.234945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d221175-5689-4fdd-94b7-0d48fa4a520c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:40.235211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d221175-5689-4fdd-94b7-0d48fa4a520c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:40.247960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d221175-5689-4fdd-94b7-0d48fa4a520c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:40.249535 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d221175-5689-4fdd-94b7-0d48fa4a520c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:40.265414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-8d221175-5689-4fdd-94b7-0d48fa4a520c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:40.265414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:40.265414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:40.265414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:40.265414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:40.265414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:40.265414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:40.265414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:40.265414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:40.265414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:40.265414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:40.265414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:40.265414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:34:40.265414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:34:40.265414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:34:40.265414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:34:40.267656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:40.267656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:40.267656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:40.267656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:40.267656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:40.267656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:40.267656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:40.267656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:40.267656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:40.267656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:40.267656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:40.267656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:40.267656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:40.267656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:40.267656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:40.267656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:40.267656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:34:40.269406 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:34:40.269406 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:40.269406 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:40.269406 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 641/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:34:40.273903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de60c7-75ff-4a3b-a8ee-0f0c8e7b4624 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:40.385539 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de60c7-75ff-4a3b-a8ee-0f0c8e7b4624 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:40.424529 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e1696-177d-4226-aa6a-b32e9de97363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:40.425143 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e1696-177d-4226-aa6a-b32e9de97363 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:40.425481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e1696-177d-4226-aa6a-b32e9de97363 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:40.425752 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e1696-177d-4226-aa6a-b32e9de97363 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:40.426427 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e1696-177d-4226-aa6a-b32e9de97363 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:40.426707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e1696-177d-4226-aa6a-b32e9de97363 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:40.428509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e1696-177d-4226-aa6a-b32e9de97363 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:40.442290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 638/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:40.448501 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d221175-5689-4fdd-94b7-0d48fa4a520c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:40.486166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:40.571738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d221175-5689-4fdd-94b7-0d48fa4a520c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:40.615425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484d115-53e7-4b1c-8b77-7eaf2eba8ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:40.616206 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484d115-53e7-4b1c-8b77-7eaf2eba8ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:40.616633 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484d115-53e7-4b1c-8b77-7eaf2eba8ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:40.616928 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484d115-53e7-4b1c-8b77-7eaf2eba8ba3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:40.628747 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484d115-53e7-4b1c-8b77-7eaf2eba8ba3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:40.630372 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484d115-53e7-4b1c-8b77-7eaf2eba8ba3 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:40.645609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-d484d115-53e7-4b1c-8b77-7eaf2eba8ba3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:34:40.645609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:40.645609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:40.645609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:40.645609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:40.645609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:40.645609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:40.645609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:40.645609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:40.645609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:40.645609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:40.645609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:40.645609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:34:40.645609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:34:40.645609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:34:40.645609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:40.647001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:40.648131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:40.648131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:40.648131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:34:40.648131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:34:40.648131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:34:40.648131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:34:40.648131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:34:40.648131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:34:40.648131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:34:40.648131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:40.648131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 642/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:40.654070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e1696-177d-4226-aa6a-b32e9de97363 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:40.659573 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:40.777646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e1696-177d-4226-aa6a-b32e9de97363 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:40.816320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b5ada-c509-4db5-ba03-6fa8a54e4319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:40.817042 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b5ada-c509-4db5-ba03-6fa8a54e4319 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:40.817285 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b5ada-c509-4db5-ba03-6fa8a54e4319 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:40.817614 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b5ada-c509-4db5-ba03-6fa8a54e4319 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:40.818491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b5ada-c509-4db5-ba03-6fa8a54e4319 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:40.819396 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b5ada-c509-4db5-ba03-6fa8a54e4319 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:40.820252 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b5ada-c509-4db5-ba03-6fa8a54e4319 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:40.842241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 639/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Aug 30 14:34:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:40.849149 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484d115-53e7-4b1c-8b77-7eaf2eba8ba3 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:40.964844 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:40.976127 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484d115-53e7-4b1c-8b77-7eaf2eba8ba3 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:41.015236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56828cc7-0743-43c8-a23a-6b9268caaeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:41.015914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56828cc7-0743-43c8-a23a-6b9268caaeb6 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:41.016218 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56828cc7-0743-43c8-a23a-6b9268caaeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:41.016503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56828cc7-0743-43c8-a23a-6b9268caaeb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:41.097740 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56828cc7-0743-43c8-a23a-6b9268caaeb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:41.100050 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56828cc7-0743-43c8-a23a-6b9268caaeb6 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:41.205245 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:41.217000 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 643/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Aug 30 14:34:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:34:41.223962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b5ada-c509-4db5-ba03-6fa8a54e4319 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:41.300909 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:41.348871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b5ada-c509-4db5-ba03-6fa8a54e4319 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:41.392787 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08726b-ebdc-4d04-8832-ba47c93cf4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:41.393437 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08726b-ebdc-4d04-8832-ba47c93cf4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:41.393718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08726b-ebdc-4d04-8832-ba47c93cf4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:41.394001 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08726b-ebdc-4d04-8832-ba47c93cf4f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:41.405552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08726b-ebdc-4d04-8832-ba47c93cf4f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:41.407040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08726b-ebdc-4d04-8832-ba47c93cf4f9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:41.422913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-db08726b-ebdc-4d04-8832-ba47c93cf4f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:41.422913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:41.422913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:41.422913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:41.422913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:41.422913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:41.422913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:41.422913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:41.422913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:41.422913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:41.422913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:41.422913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:41.422913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:34:41.422913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:34:41.422913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:34:41.422913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:34:41.424761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:41.424761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:41.424761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:41.424761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:41.424761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:41.424761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:41.424761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:41.424761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:41.424761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:41.424761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:41.424761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:41.424761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:41.424761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:41.424761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:41.424761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:41.424761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:41.424761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:34:41.426497 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:34:41.426497 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:34:41.426497 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:34:41.426497 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 640/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:34:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:41.431884 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56828cc7-0743-43c8-a23a-6b9268caaeb6 None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:41.538104 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56828cc7-0743-43c8-a23a-6b9268caaeb6 None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:41.576802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2fa7c-73d5-4f97-acf1-c16e76798a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:41.577364 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2fa7c-73d5-4f97-acf1-c16e76798a1c None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:41.577663 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2fa7c-73d5-4f97-acf1-c16e76798a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:41.577931 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2fa7c-73d5-4f97-acf1-c16e76798a1c None admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:41.578586 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2fa7c-73d5-4f97-acf1-c16e76798a1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:41.579211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2fa7c-73d5-4f97-acf1-c16e76798a1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:41.580647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2fa7c-73d5-4f97-acf1-c16e76798a1c None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:41.594665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 644/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Aug 30 14:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:41.600997 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08726b-ebdc-4d04-8832-ba47c93cf4f9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:41.712655 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08726b-ebdc-4d04-8832-ba47c93cf4f9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:41.726749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:41.756418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32646e35-469b-4de5-b06d-e6e246450e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:41.757185 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32646e35-469b-4de5-b06d-e6e246450e1b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:41.757553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32646e35-469b-4de5-b06d-e6e246450e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:41.757903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32646e35-469b-4de5-b06d-e6e246450e1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:41.765658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32646e35-469b-4de5-b06d-e6e246450e1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:41.767131 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32646e35-469b-4de5-b06d-e6e246450e1b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:41.779129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-32646e35-469b-4de5-b06d-e6e246450e1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:34:41.779129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:41.779129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:41.779129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:41.779129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:41.779129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:41.779129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:41.779129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:41.779129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:41.779129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:41.779129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:41.779129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:41.779129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:34:41.779129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:34:41.779129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:34:41.780473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:34:41.780473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:41.780473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:41.780473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:41.780473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:41.780473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:41.780473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:41.780473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:41.780473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:41.780473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:41.780473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:41.780473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:41.780473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:41.780473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:41.780473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:41.780473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:41.780473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:41.781985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:34:41.781985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:34:41.781985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:34:41.781985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:34:41.781985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:34:41.781985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:34:41.781985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:34:41.781985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 30 14:34:41.781985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 641/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Aug 30 14:34:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:34:41.787200 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2fa7c-73d5-4f97-acf1-c16e76798a1c None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:41.842888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:41.909757 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2fa7c-73d5-4f97-acf1-c16e76798a1c None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:41.959473 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a0d0d-5b68-4048-aed4-a5844c465270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:41.960271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a0d0d-5b68-4048-aed4-a5844c465270 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:41.960557 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a0d0d-5b68-4048-aed4-a5844c465270 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:41.960826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a0d0d-5b68-4048-aed4-a5844c465270 None admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:41.961688 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a0d0d-5b68-4048-aed4-a5844c465270 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:41.961994 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a0d0d-5b68-4048-aed4-a5844c465270 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:41.963744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a0d0d-5b68-4048-aed4-a5844c465270 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:41.978218 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 645/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Aug 30 14:34:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:41.984935 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32646e35-469b-4de5-b06d-e6e246450e1b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:42.116878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32646e35-469b-4de5-b06d-e6e246450e1b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:42.155948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb63ea4-3e02-438f-ad6f-46a22781cf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:42.156594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb63ea4-3e02-438f-ad6f-46a22781cf8e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:42.156869 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb63ea4-3e02-438f-ad6f-46a22781cf8e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:42.157131 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb63ea4-3e02-438f-ad6f-46a22781cf8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:42.157867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb63ea4-3e02-438f-ad6f-46a22781cf8e None admin] RBAC: Adding query filter params (role.id=67ecca18fcea44ed8f6514715dcd8db8, scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee, user.id=d7cdd6b1ee9f4e97beb9bb037e446c8f) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:42.158160 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb63ea4-3e02-438f-ad6f-46a22781cf8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:42.159620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb63ea4-3e02-438f-ad6f-46a22781cf8e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:42.166155 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 642/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Aug 30 14:34:41 2022] GET /identity/v3/role_assignments?user.id=d7cdd6b1ee9f4e97beb9bb037e446c8f&scope.project.id=1a9c57a581324217ad2e64a0c9c7fdee&role.id=67ecca18fcea44ed8f6514715dcd8db8 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:44.343141 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5ebca-4854-4049-99d4-6a5d1b67a28b None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:44.343735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5ebca-4854-4049-99d4-6a5d1b67a28b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:44.343876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5ebca-4854-4049-99d4-6a5d1b67a28b None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:44.346077 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 646/1288] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:34:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:44.352346 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022f0d1-749c-43ea-972a-89aaf2e3363f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:44.352930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022f0d1-749c-43ea-972a-89aaf2e3363f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:44.352930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022f0d1-749c-43ea-972a-89aaf2e3363f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:44.375843 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4022f0d1-749c-43ea-972a-89aaf2e3363f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:44.377123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4022f0d1-749c-43ea-972a-89aaf2e3363f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:44.414605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:44.457503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 643/1289] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:34:44.727870 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5ebca-4854-4049-99d4-6a5d1b67a28b None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:44.837415 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5ebca-4854-4049-99d4-6a5d1b67a28b None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:44.849259 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:44.874914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15783bcb-8740-47d4-a30c-bbcbe5903b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:44.875553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15783bcb-8740-47d4-a30c-bbcbe5903b0f None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:44.875832 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15783bcb-8740-47d4-a30c-bbcbe5903b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:44.876124 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15783bcb-8740-47d4-a30c-bbcbe5903b0f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:44.876715 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15783bcb-8740-47d4-a30c-bbcbe5903b0f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:44.878080 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15783bcb-8740-47d4-a30c-bbcbe5903b0f None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:44.887352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-15783bcb-8740-47d4-a30c-bbcbe5903b0f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:34:44.887352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:44.887352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:34:44.887352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:34:44.887352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:34:44.887352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:34:44.887352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:34:44.887352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:34:44.887352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:34:44.887352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:44.887352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:34:44.887352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:44.887352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:44.887352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:44.887352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:44.887352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:44.887352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:44.888529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:44.888529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:44.888529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:44.888529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:44.888529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:44.888529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:44.888529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:34:44.888529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:34:44.888529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:34:44.888529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:34:44.888529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:44.888529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:44.888529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:44.888529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:44.888529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:44.888529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:44.889536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:44.889536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:44.889536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:44.889536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:44.889536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:44.889536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:44.889536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:44.889536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:44.889536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:44.889536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:44.889536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:34:44.889536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:34:44.889536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:34:44.889536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:44.892188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 647/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Aug 30 14:34:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:44.894536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022f0d1-749c-43ea-972a-89aaf2e3363f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:45.007234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022f0d1-749c-43ea-972a-89aaf2e3363f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:45.043324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8e5f4-ea23-4cd3-82eb-17538f3b7243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:45.043939 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8e5f4-ea23-4cd3-82eb-17538f3b7243 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:45.044246 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8e5f4-ea23-4cd3-82eb-17538f3b7243 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:45.044515 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8e5f4-ea23-4cd3-82eb-17538f3b7243 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:45.045183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8e5f4-ea23-4cd3-82eb-17538f3b7243 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:45.045481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8e5f4-ea23-4cd3-82eb-17538f3b7243 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:45.046861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8e5f4-ea23-4cd3-82eb-17538f3b7243 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:45.051851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 644/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Tue Aug 30 14:34:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:45.057921 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15783bcb-8740-47d4-a30c-bbcbe5903b0f None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:45.164429 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15783bcb-8740-47d4-a30c-bbcbe5903b0f None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:45.200484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce55377-d301-4607-b305-b245a698e9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:45.201095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce55377-d301-4607-b305-b245a698e9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:45.201375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce55377-d301-4607-b305-b245a698e9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:45.201637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce55377-d301-4607-b305-b245a698e9d3 None admin] PATH_INFO: `/v3/services` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:45.202266 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce55377-d301-4607-b305-b245a698e9d3 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:45.202561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce55377-d301-4607-b305-b245a698e9d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:45.204311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce55377-d301-4607-b305-b245a698e9d3 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:45.208955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 648/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Tue Aug 30 14:34:45 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:47.182359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e9331-cd69-48cf-900a-1feafb016f79 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:47.182965 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e9331-cd69-48cf-900a-1feafb016f79 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:47.183056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e9331-cd69-48cf-900a-1feafb016f79 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:47.184439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 645/1293] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:47.189948 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71f8a1-245b-4f7e-a797-6fe4bc2b8e70 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:47.190345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71f8a1-245b-4f7e-a797-6fe4bc2b8e70 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:47.190487 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71f8a1-245b-4f7e-a797-6fe4bc2b8e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:47.210365 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-3a71f8a1-245b-4f7e-a797-6fe4bc2b8e70 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:47.211560 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-3a71f8a1-245b-4f7e-a797-6fe4bc2b8e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:47.242587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:47.283321 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 649/1294] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:34:47.552253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e9331-cd69-48cf-900a-1feafb016f79 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:47.585020 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:47.666478 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e9331-cd69-48cf-900a-1feafb016f79 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:47.706050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e24a03-6ec8-4dde-a355-b5202ac5100e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:47.706672 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e24a03-6ec8-4dde-a355-b5202ac5100e None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:47.706940 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e24a03-6ec8-4dde-a355-b5202ac5100e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:47.707197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e24a03-6ec8-4dde-a355-b5202ac5100e None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:47.707854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e24a03-6ec8-4dde-a355-b5202ac5100e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:47.709263 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e24a03-6ec8-4dde-a355-b5202ac5100e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:47.726303 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 646/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Aug 30 14:34:47 2022] POST /identity/v3/services => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:34:49.661675 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcd85f-8a75-4d77-9932-eebb0a43bbab None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:49.662217 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcd85f-8a75-4d77-9932-eebb0a43bbab None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:49.662217 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcd85f-8a75-4d77-9932-eebb0a43bbab None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:49.663565 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 650/1296] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:49.669003 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444abca4-c1ab-4939-908a-6bdb8af7d281 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:49.669437 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444abca4-c1ab-4939-908a-6bdb8af7d281 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:49.669527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444abca4-c1ab-4939-908a-6bdb8af7d281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:49.689070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-444abca4-c1ab-4939-908a-6bdb8af7d281 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:49.690272 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-444abca4-c1ab-4939-908a-6bdb8af7d281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:49.786517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 647/1297] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:34:50.064637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcd85f-8a75-4d77-9932-eebb0a43bbab None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:50.177631 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcd85f-8a75-4d77-9932-eebb0a43bbab None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:50.214243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3815155-9e34-4525-b3b9-29ed4ec3d4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:50.214877 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3815155-9e34-4525-b3b9-29ed4ec3d4fe None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:50.215150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3815155-9e34-4525-b3b9-29ed4ec3d4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:50.215419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3815155-9e34-4525-b3b9-29ed4ec3d4fe None admin] PATH_INFO: `/v3/services/placement` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:50.216024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3815155-9e34-4525-b3b9-29ed4ec3d4fe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:50.217399 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3815155-9e34-4525-b3b9-29ed4ec3d4fe None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:50.226698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-c3815155-9e34-4525-b3b9-29ed4ec3d4fe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:34:50.226698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:50.226698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:34:50.226698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:34:50.226698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:34:50.226698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:34:50.226698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:34:50.226698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:34:50.226698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:34:50.226698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:50.226698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:34:50.226698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:50.226698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:50.226698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:50.226698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:50.226698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:50.226698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:50.227752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:50.227752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:50.227752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:50.227752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:50.227752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:50.227752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:50.227752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:34:50.227752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:34:50.227752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:34:50.227752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:34:50.227752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:50.227752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:50.227752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:50.227752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:50.227752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:50.227752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:50.228914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:50.228914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:50.228914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:50.228914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:50.228914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:50.228914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:50.228914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:50.228914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:50.228914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:50.228914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:50.228914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:34:50.228914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:34:50.228914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:34:50.228914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:50.228914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 651/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Aug 30 14:34:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:50.236327 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444abca4-c1ab-4939-908a-6bdb8af7d281 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:50.254158 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:50.341289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444abca4-c1ab-4939-908a-6bdb8af7d281 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:50.378640 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb0cc3-2479-4e28-ad20-4d666b186a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:50.379319 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb0cc3-2479-4e28-ad20-4d666b186a5b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:50.379594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb0cc3-2479-4e28-ad20-4d666b186a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:50.379870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb0cc3-2479-4e28-ad20-4d666b186a5b None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:50.380606 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb0cc3-2479-4e28-ad20-4d666b186a5b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:50.380911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb0cc3-2479-4e28-ad20-4d666b186a5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:50.382508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb0cc3-2479-4e28-ad20-4d666b186a5b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:50.389234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 648/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Tue Aug 30 14:34:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:50.396544 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3815155-9e34-4525-b3b9-29ed4ec3d4fe None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:50.434960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:50.517395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3815155-9e34-4525-b3b9-29ed4ec3d4fe None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:50.559325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32586863-d4e2-405a-acf3-e8ef08b14bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:50.559980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32586863-d4e2-405a-acf3-e8ef08b14bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:50.560770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32586863-d4e2-405a-acf3-e8ef08b14bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:50.560770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32586863-d4e2-405a-acf3-e8ef08b14bc1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:50.561525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32586863-d4e2-405a-acf3-e8ef08b14bc1 None admin] RBAC: Adding query filter params (interface=public, service_id=d5c95cf319414bffbeb3deabe2a9617d, region_id=RegionOne) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:50.561825 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32586863-d4e2-405a-acf3-e8ef08b14bc1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:50.563233 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32586863-d4e2-405a-acf3-e8ef08b14bc1 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:50.569374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 652/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Aug 30 14:34:50 2022] GET /identity/v3/endpoints?service_id=d5c95cf319414bffbeb3deabe2a9617d&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:50.575740 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb0cc3-2479-4e28-ad20-4d666b186a5b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:50.671881 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:50.696960 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb0cc3-2479-4e28-ad20-4d666b186a5b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:50.736374 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912aca7e-08d8-46b5-903f-de38aa6df125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:50.736991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912aca7e-08d8-46b5-903f-de38aa6df125 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:50.737251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912aca7e-08d8-46b5-903f-de38aa6df125 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:50.737530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912aca7e-08d8-46b5-903f-de38aa6df125 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:50.738135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912aca7e-08d8-46b5-903f-de38aa6df125 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:50.742130 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912aca7e-08d8-46b5-903f-de38aa6df125 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:50.755910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 649/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Aug 30 14:34:50 2022] GET /identity/v3/services => generated 2191 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:34:52.778349 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bccbd-0812-48f6-b63e-9f58ae0adf46 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:52.778942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bccbd-0812-48f6-b63e-9f58ae0adf46 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:52.778942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bccbd-0812-48f6-b63e-9f58ae0adf46 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:52.780292 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 653/1302] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:52.788271 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91d326-bc53-42ba-9e3c-655e1a96cbb9 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:52.789023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91d326-bc53-42ba-9e3c-655e1a96cbb9 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:52.789023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91d326-bc53-42ba-9e3c-655e1a96cbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:52.810879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4d91d326-bc53-42ba-9e3c-655e1a96cbb9 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:52.812625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4d91d326-bc53-42ba-9e3c-655e1a96cbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:52.892497 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 650/1303] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:52 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:34:53.177967 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bccbd-0812-48f6-b63e-9f58ae0adf46 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:53.278947 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:53.287137 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bccbd-0812-48f6-b63e-9f58ae0adf46 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:53.326663 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c46ce8-a153-4c6c-99b2-793d4856b49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:53.327298 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c46ce8-a153-4c6c-99b2-793d4856b49e None admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:53.327570 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c46ce8-a153-4c6c-99b2-793d4856b49e None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:53.327834 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c46ce8-a153-4c6c-99b2-793d4856b49e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:53.328460 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c46ce8-a153-4c6c-99b2-793d4856b49e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:53.330248 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c46ce8-a153-4c6c-99b2-793d4856b49e None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:53.340040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application [None req-57c46ce8-a153-4c6c-99b2-793d4856b49e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:34:53.340040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:53.340040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:34:53.340040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:34:53.340040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:34:53.340040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:34:53.340040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:34:53.340040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:34:53.340040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:34:53.340040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:53.340040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:34:53.340040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:53.340040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:34:53.340040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:34:53.340040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:34:53.340040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:34:53.340040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:34:53.341191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:34:53.341191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:34:53.341191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:34:53.341191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:34:53.341191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:34:53.341191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:34:53.341191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:34:53.341191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:34:53.341191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:34:53.341191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:34:53.341191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:34:53.341191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:34:53.341191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:34:53.341191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:34:53.341191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:34:53.341191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:34:53.342156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:34:53.342156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application with Lock( Aug 30 14:34:53.342156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:34:53.342156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:34:53.342156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:34:53.342156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:34:53.342156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:34:53.342156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:34:53.342156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:34:53.342156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:34:53.342156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:34:53.342156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:34:53.342156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:34:53.342156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: ERROR keystone.server.flask.application Aug 30 14:34:53.342156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 654/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Aug 30 14:34:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:34:53.347596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91d326-bc53-42ba-9e3c-655e1a96cbb9 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:53.428637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:53.458246 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91d326-bc53-42ba-9e3c-655e1a96cbb9 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:53.499368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe2e03c-6285-4111-80b1-1772028ba9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:53.500026 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2e03c-6285-4111-80b1-1772028ba9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:53.500303 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2e03c-6285-4111-80b1-1772028ba9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:53.500572 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2e03c-6285-4111-80b1-1772028ba9d4 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:53.501216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe2e03c-6285-4111-80b1-1772028ba9d4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:34:53.501519 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe2e03c-6285-4111-80b1-1772028ba9d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:53.502920 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe2e03c-6285-4111-80b1-1772028ba9d4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:53.508677 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 651/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Tue Aug 30 14:34:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:34:53.515166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c46ce8-a153-4c6c-99b2-793d4856b49e None admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:53.623143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c46ce8-a153-4c6c-99b2-793d4856b49e None admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:53.662762 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1812ce6-622d-4290-8023-32ae2a2d4b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:53.663524 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1812ce6-622d-4290-8023-32ae2a2d4b27 None admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:53.663974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1812ce6-622d-4290-8023-32ae2a2d4b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:53.664445 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1812ce6-622d-4290-8023-32ae2a2d4b27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:53.665556 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1812ce6-622d-4290-8023-32ae2a2d4b27 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:53.667701 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1812ce6-622d-4290-8023-32ae2a2d4b27 None admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:53.701432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 655/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:34:53 2022] POST /identity/v3/endpoints => generated 331 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:35:03.505338 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=30 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:03.688987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:01.018659 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569718f-9e42-423a-a542-b1a58152d879 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:01.019563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569718f-9e42-423a-a542-b1a58152d879 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:01.019979 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569718f-9e42-423a-a542-b1a58152d879 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:01.027260 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f9d7a-2ca2-4619-89b2-fec96a92431e None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:01.028145 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f9d7a-2ca2-4619-89b2-fec96a92431e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:01.028564 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f9d7a-2ca2-4619-89b2-fec96a92431e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:01.030774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 652/1307] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:36:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:01.118999 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-361f9d7a-2ca2-4619-89b2-fec96a92431e None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:01.120569 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-361f9d7a-2ca2-4619-89b2-fec96a92431e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:01.286433 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 656/1308] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:01.620260 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b05ad-435c-42db-a2ef-79d7c7e865cd None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:01.620643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b05ad-435c-42db-a2ef-79d7c7e865cd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:01.621397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b05ad-435c-42db-a2ef-79d7c7e865cd None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:01.622374 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 653/1309] 10.222.0.25 () {54 vars in 965 bytes} [Tue Aug 30 14:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:01.629684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86477a-c42e-48cc-ac50-01abfd2084d1 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:01.630002 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86477a-c42e-48cc-ac50-01abfd2084d1 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:01.630184 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86477a-c42e-48cc-ac50-01abfd2084d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:01.733464 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-8c86477a-c42e-48cc-ac50-01abfd2084d1 None None] MFA Rules not processed for user `932fce741c5a4deba79a310a14e6bd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:01.735052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-8c86477a-c42e-48cc-ac50-01abfd2084d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:01.760053 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:01.826567 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 657/1310] 10.222.0.25 () {58 vars in 1062 bytes} [Tue Aug 30 14:36:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:01.835955 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b05ad-435c-42db-a2ef-79d7c7e865cd None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:01.837967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a22b05ad-435c-42db-a2ef-79d7c7e865cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:01.955768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b05ad-435c-42db-a2ef-79d7c7e865cd None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:02.037304 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab627fa-80a2-41ea-bd74-6918ef288f51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:02.038426 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab627fa-80a2-41ea-bd74-6918ef288f51 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:02.038794 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab627fa-80a2-41ea-bd74-6918ef288f51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:02.039178 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab627fa-80a2-41ea-bd74-6918ef288f51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:02.041232 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-eab627fa-80a2-41ea-bd74-6918ef288f51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:02.102276 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:02.161008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab627fa-80a2-41ea-bd74-6918ef288f51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:02.162499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab627fa-80a2-41ea-bd74-6918ef288f51 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:02.182854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 654/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:04.411181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27c200-7bf4-40c2-a1fd-cb4c36fcb17d None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:04.411775 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27c200-7bf4-40c2-a1fd-cb4c36fcb17d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:04.411775 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27c200-7bf4-40c2-a1fd-cb4c36fcb17d None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:04.413103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 658/1312] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:04.420898 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a06aad-19e6-4053-bad7-f17ff5e2018c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:04.421420 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a06aad-19e6-4053-bad7-f17ff5e2018c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:04.421420 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a06aad-19e6-4053-bad7-f17ff5e2018c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:04.496961 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f6a06aad-19e6-4053-bad7-f17ff5e2018c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:04.498239 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f6a06aad-19e6-4053-bad7-f17ff5e2018c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:04.598286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 655/1313] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:36:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:04.876450 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27c200-7bf4-40c2-a1fd-cb4c36fcb17d None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:04.878036 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-9f27c200-7bf4-40c2-a1fd-cb4c36fcb17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:04.984730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27c200-7bf4-40c2-a1fd-cb4c36fcb17d None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:05.059053 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:05.061906 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a782e-b610-4900-8213-8f95dafc26c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:05.062558 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a782e-b610-4900-8213-8f95dafc26c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:05.062836 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a782e-b610-4900-8213-8f95dafc26c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:05.063109 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a782e-b610-4900-8213-8f95dafc26c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:05.065558 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-607a782e-b610-4900-8213-8f95dafc26c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:05.184582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a782e-b610-4900-8213-8f95dafc26c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:05.185907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a782e-b610-4900-8213-8f95dafc26c3 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:05.207620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 659/1314] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:07.486556 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfdabe-330c-4f72-b700-37a630db0244 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:07.487111 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfdabe-330c-4f72-b700-37a630db0244 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:07.487111 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfdabe-330c-4f72-b700-37a630db0244 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:07.488380 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 656/1315] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:07.493888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352e059-f951-41d7-a384-3d2ab1f6fab6 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:07.494390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352e059-f951-41d7-a384-3d2ab1f6fab6 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:07.494502 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352e059-f951-41d7-a384-3d2ab1f6fab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:07.567284 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-9352e059-f951-41d7-a384-3d2ab1f6fab6 None None] MFA Rules not processed for user `7ded74de77c442b1853b2349fb3567ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:07.568524 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-9352e059-f951-41d7-a384-3d2ab1f6fab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:07.632153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:07.679844 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 660/1316] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:36:07 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:07.975891 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca91ec-7337-41ab-bfcf-f6ae35cf044c None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:07.976153 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca91ec-7337-41ab-bfcf-f6ae35cf044c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:07.976452 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca91ec-7337-41ab-bfcf-f6ae35cf044c None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:07.977797 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 657/1317] 10.222.0.25 () {54 vars in 965 bytes} [Tue Aug 30 14:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:07.984055 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd701397-5fbf-4adc-9dd8-2b3672910387 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:07.984377 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd701397-5fbf-4adc-9dd8-2b3672910387 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:07.984644 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd701397-5fbf-4adc-9dd8-2b3672910387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:08.065586 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-fd701397-5fbf-4adc-9dd8-2b3672910387 None None] MFA Rules not processed for user `932fce741c5a4deba79a310a14e6bd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:08.066799 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-fd701397-5fbf-4adc-9dd8-2b3672910387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:08.154977 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 661/1318] 10.222.0.25 () {58 vars in 1062 bytes} [Tue Aug 30 14:36:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:08.167314 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca91ec-7337-41ab-bfcf-f6ae35cf044c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:08.169791 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d1ca91ec-7337-41ab-bfcf-f6ae35cf044c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:08.180318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:08.290795 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca91ec-7337-41ab-bfcf-f6ae35cf044c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:08.377113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a77c4-4d37-4f2d-ba02-fdc1be67ff99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:08.378078 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a77c4-4d37-4f2d-ba02-fdc1be67ff99 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:08.378566 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a77c4-4d37-4f2d-ba02-fdc1be67ff99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:08.379049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a77c4-4d37-4f2d-ba02-fdc1be67ff99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:08.382289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7d7a77c4-4d37-4f2d-ba02-fdc1be67ff99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:08.455100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:08.504313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a77c4-4d37-4f2d-ba02-fdc1be67ff99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:08.505758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a77c4-4d37-4f2d-ba02-fdc1be67ff99 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:08.526075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 658/1319] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:08 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:11.413896 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb4507-f7da-463c-8149-735d3a2d9c22 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:11.414495 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb4507-f7da-463c-8149-735d3a2d9c22 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:11.414495 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb4507-f7da-463c-8149-735d3a2d9c22 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:11.415793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 662/1320] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:11.421759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b011ab-04a5-4f4f-82ed-c01f453344ad None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:11.422327 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b011ab-04a5-4f4f-82ed-c01f453344ad None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:11.422327 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b011ab-04a5-4f4f-82ed-c01f453344ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:11.492438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-99b011ab-04a5-4f4f-82ed-c01f453344ad None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:11.493673 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-99b011ab-04a5-4f4f-82ed-c01f453344ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:11.631535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 659/1321] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:36:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:11.911028 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb4507-f7da-463c-8149-735d3a2d9c22 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:11.912649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-8fcb4507-f7da-463c-8149-735d3a2d9c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:11.972082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:12.029403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb4507-f7da-463c-8149-735d3a2d9c22 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:12.113480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98162124-111b-4c56-9086-c987ff590082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:12.114098 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98162124-111b-4c56-9086-c987ff590082 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:12.114374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98162124-111b-4c56-9086-c987ff590082 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:12.114658 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98162124-111b-4c56-9086-c987ff590082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:12.116601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-98162124-111b-4c56-9086-c987ff590082 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:12.231419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:12.246979 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98162124-111b-4c56-9086-c987ff590082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:12.248571 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98162124-111b-4c56-9086-c987ff590082 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:12.269731 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 663/1322] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:15.033025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081a6ed-e64e-4e16-af94-4af8ce98136c None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:15.033609 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081a6ed-e64e-4e16-af94-4af8ce98136c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:15.033880 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081a6ed-e64e-4e16-af94-4af8ce98136c None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:15.036023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 660/1323] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:36:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:15.047121 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba2cbd-cc57-4097-8c1b-19045c0afeb7 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:15.047798 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba2cbd-cc57-4097-8c1b-19045c0afeb7 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:15.047798 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba2cbd-cc57-4097-8c1b-19045c0afeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:15.122172 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-15ba2cbd-cc57-4097-8c1b-19045c0afeb7 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:15.123793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-15ba2cbd-cc57-4097-8c1b-19045c0afeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:15.242988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 664/1324] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:15.538311 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081a6ed-e64e-4e16-af94-4af8ce98136c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:15.540034 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b081a6ed-e64e-4e16-af94-4af8ce98136c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:15.566052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:15.681894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081a6ed-e64e-4e16-af94-4af8ce98136c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:15.771926 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b0c14-eefd-49fd-b206-4b765a2dc88b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:15.772479 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b0c14-eefd-49fd-b206-4b765a2dc88b service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:15.772751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b0c14-eefd-49fd-b206-4b765a2dc88b service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:15.772985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b0c14-eefd-49fd-b206-4b765a2dc88b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:15.774652 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8f2b0c14-eefd-49fd-b206-4b765a2dc88b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:15.853548 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:15.906363 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2b0c14-eefd-49fd-b206-4b765a2dc88b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:15.909163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2b0c14-eefd-49fd-b206-4b765a2dc88b service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:15.937774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 661/1325] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:17.955716 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53322023-6669-4dee-a4f5-2ea80302e449 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:17.956369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53322023-6669-4dee-a4f5-2ea80302e449 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:17.956369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53322023-6669-4dee-a4f5-2ea80302e449 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:17.957649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 665/1326] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:17.963728 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632c7d9-a443-46aa-8f51-8131ab07eb83 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:17.964395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632c7d9-a443-46aa-8f51-8131ab07eb83 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:17.964395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632c7d9-a443-46aa-8f51-8131ab07eb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:18.040370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0632c7d9-a443-46aa-8f51-8131ab07eb83 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:18.041593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0632c7d9-a443-46aa-8f51-8131ab07eb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:18.148519 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 662/1327] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:18.430342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53322023-6669-4dee-a4f5-2ea80302e449 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:18.431922 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-53322023-6669-4dee-a4f5-2ea80302e449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:18.516294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:18.569585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53322023-6669-4dee-a4f5-2ea80302e449 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:18.655946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c36ec6-eae9-4482-8982-786162e6c1b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:18.656596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c36ec6-eae9-4482-8982-786162e6c1b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:18.656869 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c36ec6-eae9-4482-8982-786162e6c1b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:18.657128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c36ec6-eae9-4482-8982-786162e6c1b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:18.658773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-11c36ec6-eae9-4482-8982-786162e6c1b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:18.785903 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:18.788760 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c36ec6-eae9-4482-8982-786162e6c1b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:18.790673 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c36ec6-eae9-4482-8982-786162e6c1b8 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:18.812230 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 666/1328] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:20.816733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e90d1f-5df2-4ec2-9958-6a125bbcc018 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:20.817397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e90d1f-5df2-4ec2-9958-6a125bbcc018 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:20.817397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e90d1f-5df2-4ec2-9958-6a125bbcc018 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:20.818663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 663/1329] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:20.825420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3b30d-2203-4d4f-85b7-bb6505a04d26 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:20.826001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3b30d-2203-4d4f-85b7-bb6505a04d26 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:20.826204 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3b30d-2203-4d4f-85b7-bb6505a04d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:20.901051 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-2bd3b30d-2203-4d4f-85b7-bb6505a04d26 None None] MFA Rules not processed for user `7ded74de77c442b1853b2349fb3567ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:20.902208 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2bd3b30d-2203-4d4f-85b7-bb6505a04d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:21.021693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 667/1330] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:36:20 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:21.305817 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e90d1f-5df2-4ec2-9958-6a125bbcc018 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:21.307500 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-86e90d1f-5df2-4ec2-9958-6a125bbcc018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:21.347582 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:21.434937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e90d1f-5df2-4ec2-9958-6a125bbcc018 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:21.537419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de53af4f-edd7-4308-afeb-10451aeae62a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:21.537917 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53af4f-edd7-4308-afeb-10451aeae62a service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:21.538189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53af4f-edd7-4308-afeb-10451aeae62a service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:21.538445 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53af4f-edd7-4308-afeb-10451aeae62a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:21.540112 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-de53af4f-edd7-4308-afeb-10451aeae62a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:21.623355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:21.657356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de53af4f-edd7-4308-afeb-10451aeae62a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:21.658821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de53af4f-edd7-4308-afeb-10451aeae62a service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:21.677041 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 664/1331] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:21 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:36:23.894772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2c360-afa3-49ce-bbed-51b8f896d446 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:23.895339 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2c360-afa3-49ce-bbed-51b8f896d446 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:23.895339 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2c360-afa3-49ce-bbed-51b8f896d446 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:23.896626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 668/1332] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:23.902003 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a58b1-ad62-43cd-a10f-f05fed92cd7c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:23.902471 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a58b1-ad62-43cd-a10f-f05fed92cd7c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:23.902585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a58b1-ad62-43cd-a10f-f05fed92cd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:23.984681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-226a58b1-ad62-43cd-a10f-f05fed92cd7c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:23.985922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-226a58b1-ad62-43cd-a10f-f05fed92cd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:24.089261 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 665/1333] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:24.374237 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2c360-afa3-49ce-bbed-51b8f896d446 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:24.375836 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7aa2c360-afa3-49ce-bbed-51b8f896d446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:24.460075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:24.500314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2c360-afa3-49ce-bbed-51b8f896d446 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:24.589965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd01026-3329-4f03-afcf-e6e98973eac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:24.590504 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd01026-3329-4f03-afcf-e6e98973eac6 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:24.590781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd01026-3329-4f03-afcf-e6e98973eac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:24.591049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd01026-3329-4f03-afcf-e6e98973eac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:24.593333 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5fd01026-3329-4f03-afcf-e6e98973eac6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:24.755364 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd01026-3329-4f03-afcf-e6e98973eac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:24.758365 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd01026-3329-4f03-afcf-e6e98973eac6 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:24.778960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:24.790505 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 669/1334] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:26.905008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536e12b-30d9-440c-ae5f-ad67fc6a893d None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:26.905591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536e12b-30d9-440c-ae5f-ad67fc6a893d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:26.905591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536e12b-30d9-440c-ae5f-ad67fc6a893d None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:26.906927 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 666/1335] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:26.913047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1afd4c-e9b4-4b76-8b2b-fd0395b35f5f None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:26.913580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1afd4c-e9b4-4b76-8b2b-fd0395b35f5f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:26.913711 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1afd4c-e9b4-4b76-8b2b-fd0395b35f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:27.002229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-3f1afd4c-e9b4-4b76-8b2b-fd0395b35f5f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:27.003564 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-3f1afd4c-e9b4-4b76-8b2b-fd0395b35f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:27.116283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 670/1336] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:27.411715 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536e12b-30d9-440c-ae5f-ad67fc6a893d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:27.413383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8536e12b-30d9-440c-ae5f-ad67fc6a893d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:27.459873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:27.533555 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536e12b-30d9-440c-ae5f-ad67fc6a893d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:27.622276 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56141128-c2cc-43aa-967a-9094bee90864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:27.623231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56141128-c2cc-43aa-967a-9094bee90864 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:27.623747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56141128-c2cc-43aa-967a-9094bee90864 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:27.624093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56141128-c2cc-43aa-967a-9094bee90864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:27.626054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-56141128-c2cc-43aa-967a-9094bee90864 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:27.731733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:27.763433 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56141128-c2cc-43aa-967a-9094bee90864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:27.765104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56141128-c2cc-43aa-967a-9094bee90864 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:27.788246 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 667/1337] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:30.285056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18e831-9431-43f9-bbb7-31d7f07af922 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:30.285689 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18e831-9431-43f9-bbb7-31d7f07af922 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:30.285689 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18e831-9431-43f9-bbb7-31d7f07af922 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:30.287006 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 671/1338] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:30.293572 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16763064-b6a7-439d-9018-db26804ee68f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:30.293572 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16763064-b6a7-439d-9018-db26804ee68f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:30.293572 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16763064-b6a7-439d-9018-db26804ee68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:30.367557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-16763064-b6a7-439d-9018-db26804ee68f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:30.368839 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-16763064-b6a7-439d-9018-db26804ee68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:30.473595 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 668/1339] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:36:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:30.757755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18e831-9431-43f9-bbb7-31d7f07af922 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:30.759325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-ff18e831-9431-43f9-bbb7-31d7f07af922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:30.855916 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:30.890315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18e831-9431-43f9-bbb7-31d7f07af922 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:30.998633 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9713f-e784-4394-8430-5b271a471921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:30.999686 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9713f-e784-4394-8430-5b271a471921 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:31.000189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9713f-e784-4394-8430-5b271a471921 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:31.000632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9713f-e784-4394-8430-5b271a471921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:31.003718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-20c9713f-e784-4394-8430-5b271a471921 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:31.143599 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9713f-e784-4394-8430-5b271a471921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:31.151723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9713f-e784-4394-8430-5b271a471921 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:31.174318 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 672/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:35.440952 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979f11d-8a84-404b-9045-efb81a1da708 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:35.441543 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979f11d-8a84-404b-9045-efb81a1da708 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:35.441543 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979f11d-8a84-404b-9045-efb81a1da708 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:35.442846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 669/1341] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:35.448490 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f7759-4071-44cc-93d1-f03c4fb7519d None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:35.448937 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f7759-4071-44cc-93d1-f03c4fb7519d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:35.449047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f7759-4071-44cc-93d1-f03c4fb7519d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:35.455312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:35.523764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-ae9f7759-4071-44cc-93d1-f03c4fb7519d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:35.525033 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-ae9f7759-4071-44cc-93d1-f03c4fb7519d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:35.648544 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 673/1342] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:35.931771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979f11d-8a84-404b-9045-efb81a1da708 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:35.933829 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b979f11d-8a84-404b-9045-efb81a1da708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:35.998388 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:36.064122 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979f11d-8a84-404b-9045-efb81a1da708 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:36.156023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e438ca3-b386-425e-add9-5e60720a5746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:36.157001 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e438ca3-b386-425e-add9-5e60720a5746 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:36.157286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e438ca3-b386-425e-add9-5e60720a5746 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:36.157557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e438ca3-b386-425e-add9-5e60720a5746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:36.159317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3e438ca3-b386-425e-add9-5e60720a5746 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:36.282330 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:36.305673 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e438ca3-b386-425e-add9-5e60720a5746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:36.307190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e438ca3-b386-425e-add9-5e60720a5746 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:36.330820 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 670/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:39.248584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a364e-4da1-4267-a72b-3a226064014b None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:39.249305 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a364e-4da1-4267-a72b-3a226064014b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:39.249305 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a364e-4da1-4267-a72b-3a226064014b None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:39.250484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 674/1344] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:36:39.256208 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101590c6-e3c2-42bd-b3ce-4b69ddbf30ff None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:39.256743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101590c6-e3c2-42bd-b3ce-4b69ddbf30ff None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:39.256947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101590c6-e3c2-42bd-b3ce-4b69ddbf30ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:39.332571 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-101590c6-e3c2-42bd-b3ce-4b69ddbf30ff None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:39.333654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-101590c6-e3c2-42bd-b3ce-4b69ddbf30ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:39.475747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 671/1345] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:36:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:39.989900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a364e-4da1-4267-a72b-3a226064014b None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:39.991513 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-024a364e-4da1-4267-a72b-3a226064014b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:40.123904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:40.158852 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a364e-4da1-4267-a72b-3a226064014b None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:40.286182 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67910ab9-9cf1-4428-93b5-789071a25950 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:40.287257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67910ab9-9cf1-4428-93b5-789071a25950 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:40.287631 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67910ab9-9cf1-4428-93b5-789071a25950 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:40.288161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67910ab9-9cf1-4428-93b5-789071a25950 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:40.291423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-67910ab9-9cf1-4428-93b5-789071a25950 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:40.487874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67910ab9-9cf1-4428-93b5-789071a25950 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:40.489442 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67910ab9-9cf1-4428-93b5-789071a25950 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:40.520305 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 675/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:45.050154 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a8b63-fe54-4299-a3c0-6a8bb2126a6e None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:45.050644 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a8b63-fe54-4299-a3c0-6a8bb2126a6e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:45.050738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a8b63-fe54-4299-a3c0-6a8bb2126a6e None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:45.052187 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 672/1347] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:45.058295 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa55203-8a0f-49ac-91b4-0a0fbe3afe2c None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:45.058966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa55203-8a0f-49ac-91b4-0a0fbe3afe2c None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:45.058966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa55203-8a0f-49ac-91b4-0a0fbe3afe2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:45.076566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:45.139785 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-3fa55203-8a0f-49ac-91b4-0a0fbe3afe2c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:45.141098 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-3fa55203-8a0f-49ac-91b4-0a0fbe3afe2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:45.254068 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 676/1348] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:45.542267 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a8b63-fe54-4299-a3c0-6a8bb2126a6e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:45.544278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-674a8b63-fe54-4299-a3c0-6a8bb2126a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:45.611590 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:45.666854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a8b63-fe54-4299-a3c0-6a8bb2126a6e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:45.760063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b600798-f117-4f1d-9927-d62102c5ccdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:45.760711 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b600798-f117-4f1d-9927-d62102c5ccdc service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:45.760984 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b600798-f117-4f1d-9927-d62102c5ccdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:45.761225 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b600798-f117-4f1d-9927-d62102c5ccdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:45.763409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3b600798-f117-4f1d-9927-d62102c5ccdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:45.878232 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:45.881985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b600798-f117-4f1d-9927-d62102c5ccdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:45.883587 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b600798-f117-4f1d-9927-d62102c5ccdc service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:45.903874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 673/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:51.203803 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7adad4-5e4e-4891-ab65-c7bfb32798a2 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:51.204634 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7adad4-5e4e-4891-ab65-c7bfb32798a2 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:51.204634 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7adad4-5e4e-4891-ab65-c7bfb32798a2 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:51.207393 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 677/1350] 10.222.0.25 () {54 vars in 921 bytes} [Tue Aug 30 14:36:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:51.217283 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72781a1d-8ae2-40e0-a0b9-f765facbee3f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:51.219937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72781a1d-8ae2-40e0-a0b9-f765facbee3f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:51.219937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72781a1d-8ae2-40e0-a0b9-f765facbee3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:51.379095 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-72781a1d-8ae2-40e0-a0b9-f765facbee3f None None] MFA Rules not processed for user `d7cdd6b1ee9f4e97beb9bb037e446c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:51.380969 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-72781a1d-8ae2-40e0-a0b9-f765facbee3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:51.543746 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 674/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Aug 30 14:36:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:55.252201 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=59 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:56.710185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8080414-3a8f-4821-8aa9-07f30dbb4d29 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:56.711219 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8080414-3a8f-4821-8aa9-07f30dbb4d29 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:56.711651 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8080414-3a8f-4821-8aa9-07f30dbb4d29 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:56.713656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 678/1352] 10.222.0.25 () {54 vars in 921 bytes} [Tue Aug 30 14:36:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:56.719548 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29669578-105c-4fe4-a7e2-9fbc69b31c5c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:56.720279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29669578-105c-4fe4-a7e2-9fbc69b31c5c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:56.720279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29669578-105c-4fe4-a7e2-9fbc69b31c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:56.800181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:56.838549 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-29669578-105c-4fe4-a7e2-9fbc69b31c5c None None] MFA Rules not processed for user `d7cdd6b1ee9f4e97beb9bb037e446c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:56.840068 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-29669578-105c-4fe4-a7e2-9fbc69b31c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:56.979884 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 675/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Aug 30 14:36:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:58.791907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951343d-dad5-4772-bafb-dac95837d258 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:58.792581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951343d-dad5-4772-bafb-dac95837d258 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:58.793614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951343d-dad5-4772-bafb-dac95837d258 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:58.796390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 679/1354] 10.222.0.25 () {54 vars in 921 bytes} [Tue Aug 30 14:36:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:58.803939 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c829182-9b0c-4615-b810-3dfa6eb4a9ac None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:58.804593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c829182-9b0c-4615-b810-3dfa6eb4a9ac None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:58.804706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c829182-9b0c-4615-b810-3dfa6eb4a9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:58.916457 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2c829182-9b0c-4615-b810-3dfa6eb4a9ac None None] MFA Rules not processed for user `d7cdd6b1ee9f4e97beb9bb037e446c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:58.917653 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2c829182-9b0c-4615-b810-3dfa6eb4a9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:58.968703 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:59.046678 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 676/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Aug 30 14:36:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:02.217033 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea697ec-6d0d-4932-9841-5138c417ed6d None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:02.217588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea697ec-6d0d-4932-9841-5138c417ed6d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:02.217588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea697ec-6d0d-4932-9841-5138c417ed6d None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:02.218817 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 680/1356] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:37:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:37:02.224441 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58a098-6a42-4a02-aae1-02788e2f36a4 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:02.225016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58a098-6a42-4a02-aae1-02788e2f36a4 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:02.225016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58a098-6a42-4a02-aae1-02788e2f36a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:02.295457 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-bf58a098-6a42-4a02-aae1-02788e2f36a4 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:02.298425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bf58a098-6a42-4a02-aae1-02788e2f36a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:02.426279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 677/1357] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:02.754945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410f8dc-15ce-41a7-96e6-12b1f595aee0 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:02.755500 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410f8dc-15ce-41a7-96e6-12b1f595aee0 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:02.756023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410f8dc-15ce-41a7-96e6-12b1f595aee0 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:02.758144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 681/1358] 10.222.0.25 () {54 vars in 962 bytes} [Tue Aug 30 14:37:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:02.766243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd4b5d-c6ca-40a1-8f04-daf862b547af None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:02.766889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd4b5d-c6ca-40a1-8f04-daf862b547af None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:02.767404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd4b5d-c6ca-40a1-8f04-daf862b547af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:02.796588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:02.869936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-01bd4b5d-c6ca-40a1-8f04-daf862b547af None None] MFA Rules not processed for user `cc95bbdafb0c4f34ad0662bc2e3ba4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:02.871398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-01bd4b5d-c6ca-40a1-8f04-daf862b547af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:03.027496 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 678/1359] 10.222.0.25 () {58 vars in 1059 bytes} [Tue Aug 30 14:37:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:03.036530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410f8dc-15ce-41a7-96e6-12b1f595aee0 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:03.039423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4410f8dc-15ce-41a7-96e6-12b1f595aee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:03.222616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410f8dc-15ce-41a7-96e6-12b1f595aee0 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:03.302249 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:03.347201 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc737a-a64a-4b33-b1f7-2977261cb2fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:03.347842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc737a-a64a-4b33-b1f7-2977261cb2fa service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:03.348155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc737a-a64a-4b33-b1f7-2977261cb2fa service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:03.348390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc737a-a64a-4b33-b1f7-2977261cb2fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:03.350176 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-92bc737a-a64a-4b33-b1f7-2977261cb2fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:03.464735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc737a-a64a-4b33-b1f7-2977261cb2fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:03.466482 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc737a-a64a-4b33-b1f7-2977261cb2fa service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:03.490550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 682/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:05.696348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991a8be-7248-4e4b-af0d-be565a94604b None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:05.696955 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991a8be-7248-4e4b-af0d-be565a94604b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:05.696955 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991a8be-7248-4e4b-af0d-be565a94604b None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:05.698384 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 679/1361] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:05.705331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ec76c-0998-4513-b8ce-ce68f457c494 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:05.705881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ec76c-0998-4513-b8ce-ce68f457c494 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:05.705881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ec76c-0998-4513-b8ce-ce68f457c494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:05.797199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-979ec76c-0998-4513-b8ce-ce68f457c494 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:05.798444 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-979ec76c-0998-4513-b8ce-ce68f457c494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:05.848119 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:05.941739 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 683/1362] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:06.232840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991a8be-7248-4e4b-af0d-be565a94604b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:06.235012 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b991a8be-7248-4e4b-af0d-be565a94604b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:06.365751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:06.496258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991a8be-7248-4e4b-af0d-be565a94604b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:06.670319 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f70251-a675-4ec9-b537-ff77df6afdab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:06.670950 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f70251-a675-4ec9-b537-ff77df6afdab service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:06.671217 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f70251-a675-4ec9-b537-ff77df6afdab service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:06.671483 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f70251-a675-4ec9-b537-ff77df6afdab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:06.673200 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e1f70251-a675-4ec9-b537-ff77df6afdab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:06.705226 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:06.833154 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f70251-a675-4ec9-b537-ff77df6afdab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:06.836945 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f70251-a675-4ec9-b537-ff77df6afdab service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:06.880088 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 680/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:09.268431 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2766cf-9d49-44ab-8673-661840fdc895 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:09.269231 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2766cf-9d49-44ab-8673-661840fdc895 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:09.269231 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2766cf-9d49-44ab-8673-661840fdc895 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:09.270694 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 684/1364] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:09.279064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b2d7-1577-47da-86a0-332070913e45 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:09.279583 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b2d7-1577-47da-86a0-332070913e45 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:09.279785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b2d7-1577-47da-86a0-332070913e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:09.380387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d790b2d7-1577-47da-86a0-332070913e45 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:09.381919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d790b2d7-1577-47da-86a0-332070913e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:09.532461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:09.544566 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 681/1365] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:09.895632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45432fd-685f-4daa-89f1-f578fa8bcae3 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:09.896250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45432fd-685f-4daa-89f1-f578fa8bcae3 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:09.896748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45432fd-685f-4daa-89f1-f578fa8bcae3 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:09.899127 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 685/1366] 10.222.0.25 () {54 vars in 962 bytes} [Tue Aug 30 14:37:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:09.909245 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bfa15-1011-41c1-9802-455613ef7714 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:09.909555 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bfa15-1011-41c1-9802-455613ef7714 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:09.909972 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bfa15-1011-41c1-9802-455613ef7714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:10.020364 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ca2bfa15-1011-41c1-9802-455613ef7714 None None] MFA Rules not processed for user `cc95bbdafb0c4f34ad0662bc2e3ba4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:10.022508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ca2bfa15-1011-41c1-9802-455613ef7714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:10.227086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 682/1367] 10.222.0.25 () {58 vars in 1059 bytes} [Tue Aug 30 14:37:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:10.244515 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45432fd-685f-4daa-89f1-f578fa8bcae3 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:10.248826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b45432fd-685f-4daa-89f1-f578fa8bcae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:10.561325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45432fd-685f-4daa-89f1-f578fa8bcae3 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:10.603362 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:10.787379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e41d77-d2b1-42cc-bb10-637981a23d4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:10.788389 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e41d77-d2b1-42cc-bb10-637981a23d4a service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:10.788828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e41d77-d2b1-42cc-bb10-637981a23d4a service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:10.789300 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e41d77-d2b1-42cc-bb10-637981a23d4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:10.792696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b9e41d77-d2b1-42cc-bb10-637981a23d4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:10.949409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e41d77-d2b1-42cc-bb10-637981a23d4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:10.952804 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e41d77-d2b1-42cc-bb10-637981a23d4a service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:10.987276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 686/1368] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 744 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:11.081637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff607bf8-5b96-4916-8ebf-c7d9b598745d None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:11.082987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff607bf8-5b96-4916-8ebf-c7d9b598745d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:11.083590 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff607bf8-5b96-4916-8ebf-c7d9b598745d None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:11.088072 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 683/1369] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:37:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:11.095763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b29da-a461-49a9-99c1-67ce6af18d5b None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:11.096510 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b29da-a461-49a9-99c1-67ce6af18d5b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:11.097131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b29da-a461-49a9-99c1-67ce6af18d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:11.141323 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:11.201343 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-336b29da-a461-49a9-99c1-67ce6af18d5b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:11.203103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-336b29da-a461-49a9-99c1-67ce6af18d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:11.392050 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 687/1370] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:12.049852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46ef4d-0dec-4263-9a08-3ef85eba36e9 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:12.049852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46ef4d-0dec-4263-9a08-3ef85eba36e9 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:12.054254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46ef4d-0dec-4263-9a08-3ef85eba36e9 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:12.054254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 684/1371] 10.222.0.25 () {54 vars in 964 bytes} [Tue Aug 30 14:37:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:12.064946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e38964-00d3-4788-a2d2-e3a28a9932ce None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:12.064946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e38964-00d3-4788-a2d2-e3a28a9932ce None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:12.064946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e38964-00d3-4788-a2d2-e3a28a9932ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:12.173481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-85e38964-00d3-4788-a2d2-e3a28a9932ce None None] MFA Rules not processed for user `2788fece401a4a7ebdb516afe2dd8ec8`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:12.174737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-85e38964-00d3-4788-a2d2-e3a28a9932ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:12.279036 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:12.361702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 688/1372] 10.222.0.25 () {58 vars in 1061 bytes} [Tue Aug 30 14:37:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:37:12.386601 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46ef4d-0dec-4263-9a08-3ef85eba36e9 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:12.389640 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cd46ef4d-0dec-4263-9a08-3ef85eba36e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:12.494734 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:12.605820 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46ef4d-0dec-4263-9a08-3ef85eba36e9 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:12.754738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78964d05-3617-42b0-a9f8-a21eb3d17322 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:12.755779 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78964d05-3617-42b0-a9f8-a21eb3d17322 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:12.756255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78964d05-3617-42b0-a9f8-a21eb3d17322 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:12.756756 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78964d05-3617-42b0-a9f8-a21eb3d17322 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:12.760353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-78964d05-3617-42b0-a9f8-a21eb3d17322 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:12.955134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:12.972563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78964d05-3617-42b0-a9f8-a21eb3d17322 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:12.974069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78964d05-3617-42b0-a9f8-a21eb3d17322 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:12.998690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 685/1373] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:13.164356 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d0d86-10bd-4fa5-8154-7be06d685a8c None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:13.165066 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d0d86-10bd-4fa5-8154-7be06d685a8c None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:13.165066 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d0d86-10bd-4fa5-8154-7be06d685a8c None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:13.171420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 689/1374] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:37:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:13.172703 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c6d3d-3ec2-412c-b81c-ad2e0912b786 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:13.172958 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c6d3d-3ec2-412c-b81c-ad2e0912b786 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:13.173659 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c6d3d-3ec2-412c-b81c-ad2e0912b786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:13.246175 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0b2c6d3d-3ec2-412c-b81c-ad2e0912b786 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:13.247545 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0b2c6d3d-3ec2-412c-b81c-ad2e0912b786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:13.395277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 686/1375] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:13.687059 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d0d86-10bd-4fa5-8154-7be06d685a8c None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:13.688681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4b7d0d86-10bd-4fa5-8154-7be06d685a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:13.898485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d0d86-10bd-4fa5-8154-7be06d685a8c None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:13.974382 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:14.046553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efe0e8-c56a-48cf-89d1-2ba949c6f8b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:14.047196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efe0e8-c56a-48cf-89d1-2ba949c6f8b1 service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:14.047473 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efe0e8-c56a-48cf-89d1-2ba949c6f8b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:14.047822 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efe0e8-c56a-48cf-89d1-2ba949c6f8b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:14.049656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d4efe0e8-c56a-48cf-89d1-2ba949c6f8b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:14.186190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efe0e8-c56a-48cf-89d1-2ba949c6f8b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:14.187802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efe0e8-c56a-48cf-89d1-2ba949c6f8b1 service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:14.210887 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 690/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:16.231837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb72bd-9713-45c1-83f5-05686d08f1ac None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:16.232895 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb72bd-9713-45c1-83f5-05686d08f1ac None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:16.232895 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb72bd-9713-45c1-83f5-05686d08f1ac None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:16.233792 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 687/1377] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:16.240334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e6488-901b-4b7a-9939-59f6c948038d None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:16.240901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e6488-901b-4b7a-9939-59f6c948038d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:16.240901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e6488-901b-4b7a-9939-59f6c948038d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:16.327835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-2d5e6488-901b-4b7a-9939-59f6c948038d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:16.327835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2d5e6488-901b-4b7a-9939-59f6c948038d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:16.335041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:16.473594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 691/1378] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:16.761552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb72bd-9713-45c1-83f5-05686d08f1ac None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:16.763372 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-aaeb72bd-9713-45c1-83f5-05686d08f1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:16.860504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:16.972112 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb72bd-9713-45c1-83f5-05686d08f1ac None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:17.192302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e9aaa-1837-40f5-81ae-4e0e8d3fbf69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:17.192814 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e9aaa-1837-40f5-81ae-4e0e8d3fbf69 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:17.193096 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e9aaa-1837-40f5-81ae-4e0e8d3fbf69 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:17.193365 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e9aaa-1837-40f5-81ae-4e0e8d3fbf69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:17.195058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-474e9aaa-1837-40f5-81ae-4e0e8d3fbf69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:17.218441 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:17.312724 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e9aaa-1837-40f5-81ae-4e0e8d3fbf69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:17.315406 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e9aaa-1837-40f5-81ae-4e0e8d3fbf69 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:17.344226 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 688/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:19.522697 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76481739-796f-4e53-99ca-8f410728ad07 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:19.523753 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76481739-796f-4e53-99ca-8f410728ad07 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:19.523753 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76481739-796f-4e53-99ca-8f410728ad07 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:19.524919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 692/1380] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:37:19.530118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79e84c-6b4a-455f-9820-a0e240168974 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:19.530745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79e84c-6b4a-455f-9820-a0e240168974 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:19.530745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79e84c-6b4a-455f-9820-a0e240168974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:19.617743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9d79e84c-6b4a-455f-9820-a0e240168974 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:19.617743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9d79e84c-6b4a-455f-9820-a0e240168974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:19.740717 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:19.750372 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 689/1381] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:20.050029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76481739-796f-4e53-99ca-8f410728ad07 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:20.052166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-76481739-796f-4e53-99ca-8f410728ad07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:20.276195 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:20.300585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76481739-796f-4e53-99ca-8f410728ad07 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:20.487658 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d422cb3-492f-47b4-88cd-f932e2771e82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:20.488804 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d422cb3-492f-47b4-88cd-f932e2771e82 service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:20.489321 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d422cb3-492f-47b4-88cd-f932e2771e82 service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:20.489827 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d422cb3-492f-47b4-88cd-f932e2771e82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:20.493274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-3d422cb3-492f-47b4-88cd-f932e2771e82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:20.634999 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d422cb3-492f-47b4-88cd-f932e2771e82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:20.636708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d422cb3-492f-47b4-88cd-f932e2771e82 service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:20.647478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:20.657335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 693/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:22.812221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7c8b4-fabd-4156-96c7-c8e0d69ca7c8 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:22.812831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7c8b4-fabd-4156-96c7-c8e0d69ca7c8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:22.813263 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7c8b4-fabd-4156-96c7-c8e0d69ca7c8 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:22.815384 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 690/1383] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:37:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:22.821704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d975f-be19-4b56-9b05-902499dacf8e None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:22.822297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d975f-be19-4b56-9b05-902499dacf8e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:22.822297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d975f-be19-4b56-9b05-902499dacf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:22.897204 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-379d975f-be19-4b56-9b05-902499dacf8e None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:22.898382 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-379d975f-be19-4b56-9b05-902499dacf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:23.011372 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 694/1384] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:23.303177 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7c8b4-fabd-4156-96c7-c8e0d69ca7c8 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:23.304806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a6c7c8b4-fabd-4156-96c7-c8e0d69ca7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:23.473639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7c8b4-fabd-4156-96c7-c8e0d69ca7c8 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:23.561903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:23.625378 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e2514-004b-4ace-b104-bbf2e2d526ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:23.626274 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e2514-004b-4ace-b104-bbf2e2d526ae service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:23.626858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e2514-004b-4ace-b104-bbf2e2d526ae service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:23.627595 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e2514-004b-4ace-b104-bbf2e2d526ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:23.629862 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-be5e2514-004b-4ace-b104-bbf2e2d526ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:23.757094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e2514-004b-4ace-b104-bbf2e2d526ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:23.759441 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e2514-004b-4ace-b104-bbf2e2d526ae service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:23.788689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 691/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:25.767601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77003a-4464-4449-b684-f930f2b55cc5 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:25.768412 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77003a-4464-4449-b684-f930f2b55cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:25.768412 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77003a-4464-4449-b684-f930f2b55cc5 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:25.769588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 695/1386] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:25.775593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52042e16-b48f-46aa-9933-84a86da5d9ae None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:25.776501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52042e16-b48f-46aa-9933-84a86da5d9ae None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:25.776501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52042e16-b48f-46aa-9933-84a86da5d9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:25.859503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-52042e16-b48f-46aa-9933-84a86da5d9ae None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:25.860828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-52042e16-b48f-46aa-9933-84a86da5d9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:25.899138 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:25.981927 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 692/1387] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:26.268727 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77003a-4464-4449-b684-f930f2b55cc5 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:26.270993 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2e77003a-4464-4449-b684-f930f2b55cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:26.378359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:26.456584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77003a-4464-4449-b684-f930f2b55cc5 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:26.587586 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc96cf-fc99-46d6-80de-89830920a828 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:26.588199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc96cf-fc99-46d6-80de-89830920a828 service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:26.588699 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc96cf-fc99-46d6-80de-89830920a828 service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:26.589565 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc96cf-fc99-46d6-80de-89830920a828 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:26.591316 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-3dbc96cf-fc99-46d6-80de-89830920a828 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:26.638257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:26.696963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc96cf-fc99-46d6-80de-89830920a828 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:26.698747 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc96cf-fc99-46d6-80de-89830920a828 service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:26.720398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 696/1388] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:28.610787 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb277881-3352-4c59-a0c6-b1732aeeb502 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:28.611357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb277881-3352-4c59-a0c6-b1732aeeb502 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:28.611357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb277881-3352-4c59-a0c6-b1732aeeb502 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:28.612822 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 693/1389] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:28.618393 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed66ff-940f-4471-b343-83ca26c45b5e None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:28.618875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed66ff-940f-4471-b343-83ca26c45b5e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:28.618973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed66ff-940f-4471-b343-83ca26c45b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:28.689809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-deed66ff-940f-4471-b343-83ca26c45b5e None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:28.690966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-deed66ff-940f-4471-b343-83ca26c45b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:28.719602 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842f04c-7a91-4c3a-a924-5d1e8c8df164 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:28.720354 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842f04c-7a91-4c3a-a924-5d1e8c8df164 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:28.720354 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842f04c-7a91-4c3a-a924-5d1e8c8df164 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:28.721672 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 694/1390] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:28.726770 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f3311-ba5a-41e1-9b48-0d39ca29a8c2 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:28.727017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f3311-ba5a-41e1-9b48-0d39ca29a8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:28.727290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f3311-ba5a-41e1-9b48-0d39ca29a8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:28.795796 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 697/1391] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:28.800235 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-cd3f3311-ba5a-41e1-9b48-0d39ca29a8c2 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:28.801463 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cd3f3311-ba5a-41e1-9b48-0d39ca29a8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:28.915459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 695/1392] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:37:29.098315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efe861-a21e-4a19-8bf5-e96ab012ea3e None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:29.098595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efe861-a21e-4a19-8bf5-e96ab012ea3e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:29.098860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efe861-a21e-4a19-8bf5-e96ab012ea3e None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:29.100305 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 698/1393] 10.222.0.25 () {54 vars in 967 bytes} [Tue Aug 30 14:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:29.105482 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8905f77-90f6-42e2-bd5c-f44164644286 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:29.105770 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8905f77-90f6-42e2-bd5c-f44164644286 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:29.106036 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8905f77-90f6-42e2-bd5c-f44164644286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:29.122737 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:29.185424 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c8905f77-90f6-42e2-bd5c-f44164644286 None None] MFA Rules not processed for user `69de9368993f402cb6325e5022e064d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:29.186549 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c8905f77-90f6-42e2-bd5c-f44164644286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:29.208512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efe861-a21e-4a19-8bf5-e96ab012ea3e None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:29.210044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-09efe861-a21e-4a19-8bf5-e96ab012ea3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:29.225206 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:29.273275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 696/1394] 10.222.0.25 () {58 vars in 1064 bytes} [Tue Aug 30 14:37:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:29.282132 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8905f77-90f6-42e2-bd5c-f44164644286 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:29.292077 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8905f77-90f6-42e2-bd5c-f44164644286 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:29.377463 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89793148-2252-454a-b593-ca376087b3ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69de9368993f402cb6325e5022e064d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:29.378092 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89793148-2252-454a-b593-ca376087b3ac service glance] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:29.378368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89793148-2252-454a-b593-ca376087b3ac service glance] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:29.378633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89793148-2252-454a-b593-ca376087b3ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:29.380315 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-89793148-2252-454a-b593-ca376087b3ac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:29.388126 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efe861-a21e-4a19-8bf5-e96ab012ea3e None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:29.415110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:29.485543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:29.505241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89793148-2252-454a-b593-ca376087b3ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:29.506743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89793148-2252-454a-b593-ca376087b3ac service glance] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:29.515096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3295dc-8ccf-48b8-8478-866cc4b5de8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:29.515720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3295dc-8ccf-48b8-8478-866cc4b5de8a service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:29.516017 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3295dc-8ccf-48b8-8478-866cc4b5de8a service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:29.516288 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3295dc-8ccf-48b8-8478-866cc4b5de8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:29.518162 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-ef3295dc-8ccf-48b8-8478-866cc4b5de8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:29.528592 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 697/1395] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:29.626041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3295dc-8ccf-48b8-8478-866cc4b5de8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:29.627854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3295dc-8ccf-48b8-8478-866cc4b5de8a service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:29.649630 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 699/1396] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:37:30.306297 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a4597-ff86-4fcc-a5ea-24c1f23c3696 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:30.306834 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a4597-ff86-4fcc-a5ea-24c1f23c3696 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:30.306834 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a4597-ff86-4fcc-a5ea-24c1f23c3696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:30.386591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ad3a4597-ff86-4fcc-a5ea-24c1f23c3696 None None] MFA Rules not processed for user `8fc940594a1848aa9644b645049a1bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:30.387895 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ad3a4597-ff86-4fcc-a5ea-24c1f23c3696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:30.484776 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:30.493025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 698/1397] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Aug 30 14:37:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:30.545027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d51d6d-4ebd-46f4-9bf8-53a83c3b611d None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:30.545278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d51d6d-4ebd-46f4-9bf8-53a83c3b611d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:30.545585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d51d6d-4ebd-46f4-9bf8-53a83c3b611d None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:30.547314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 700/1398] 10.222.0.25 () {54 vars in 943 bytes} [Tue Aug 30 14:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:30.554132 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30e615-c8fe-4414-8008-a84abdc26482 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:30.554356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30e615-c8fe-4414-8008-a84abdc26482 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:30.554660 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30e615-c8fe-4414-8008-a84abdc26482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:30.643898 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7b30e615-c8fe-4414-8008-a84abdc26482 None None] MFA Rules not processed for user `d5d98f5adfaf410580481417e7ba1638`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:30.645155 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7b30e615-c8fe-4414-8008-a84abdc26482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:30.758277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 699/1399] 10.222.0.25 () {58 vars in 1040 bytes} [Tue Aug 30 14:37:30 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:30.768024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d51d6d-4ebd-46f4-9bf8-53a83c3b611d None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:30.769625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-90d51d6d-4ebd-46f4-9bf8-53a83c3b611d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:30.877049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:30.922882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d51d6d-4ebd-46f4-9bf8-53a83c3b611d None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:31.034060 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0f919-7a80-4d9c-bbbc-f03a87ecfdcf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d98f5adfaf410580481417e7ba1638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:31.034570 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0f919-7a80-4d9c-bbbc-f03a87ecfdcf service swift] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:31.034831 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0f919-7a80-4d9c-bbbc-f03a87ecfdcf service swift] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:31.035089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0f919-7a80-4d9c-bbbc-f03a87ecfdcf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:31.036792 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-37a0f919-7a80-4d9c-bbbc-f03a87ecfdcf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:31.128472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0f919-7a80-4d9c-bbbc-f03a87ecfdcf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:31.130103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0f919-7a80-4d9c-bbbc-f03a87ecfdcf service swift] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:31.133638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:31.140907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 701/1400] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Aug 30 14:37:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:37:31.620682 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd65fa1-2310-4f0e-820a-0c66b4a98bb0 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:31.621277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd65fa1-2310-4f0e-820a-0c66b4a98bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:31.621277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd65fa1-2310-4f0e-820a-0c66b4a98bb0 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:31.622609 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 700/1401] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:31.628352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f15a2-a0d8-4081-a848-938ce22d4591 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:31.628638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f15a2-a0d8-4081-a848-938ce22d4591 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:31.628913 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f15a2-a0d8-4081-a848-938ce22d4591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:31.702265 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-cd6f15a2-a0d8-4081-a848-938ce22d4591 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:31.703488 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-cd6f15a2-a0d8-4081-a848-938ce22d4591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:31.809882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 702/1402] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:32.093401 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd65fa1-2310-4f0e-820a-0c66b4a98bb0 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:32.095028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ccd65fa1-2310-4f0e-820a-0c66b4a98bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:32.163073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:32.262161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd65fa1-2310-4f0e-820a-0c66b4a98bb0 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:32.399371 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa7bfd-ff5b-44aa-bc68-b3ed3b90c7d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:32.400027 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa7bfd-ff5b-44aa-bc68-b3ed3b90c7d8 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:32.400314 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa7bfd-ff5b-44aa-bc68-b3ed3b90c7d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:32.400723 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa7bfd-ff5b-44aa-bc68-b3ed3b90c7d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:32.404493 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d4aa7bfd-ff5b-44aa-bc68-b3ed3b90c7d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:32.471085 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:32.659821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa7bfd-ff5b-44aa-bc68-b3ed3b90c7d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:32.661558 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa7bfd-ff5b-44aa-bc68-b3ed3b90c7d8 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:32.683596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 701/1403] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:34.685876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875316d7-b843-4aef-af09-a7652a2e3d6f None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:34.686472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875316d7-b843-4aef-af09-a7652a2e3d6f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:34.686472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875316d7-b843-4aef-af09-a7652a2e3d6f None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:34.692053 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 703/1404] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:34.694550 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a54d7-8a56-45bb-a93e-a09a15854113 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:34.695026 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a54d7-8a56-45bb-a93e-a09a15854113 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:34.695117 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a54d7-8a56-45bb-a93e-a09a15854113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:34.771202 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-5c5a54d7-8a56-45bb-a93e-a09a15854113 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:34.772463 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5c5a54d7-8a56-45bb-a93e-a09a15854113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:34.882031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:34.892609 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 702/1405] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:35.182924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875316d7-b843-4aef-af09-a7652a2e3d6f None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:35.184856 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-875316d7-b843-4aef-af09-a7652a2e3d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:35.287585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:35.380946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875316d7-b843-4aef-af09-a7652a2e3d6f None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:35.513826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22f0ae-23b2-46a7-923f-2ac1d2b6fc9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:35.514516 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22f0ae-23b2-46a7-923f-2ac1d2b6fc9e service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:35.514662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22f0ae-23b2-46a7-923f-2ac1d2b6fc9e service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:35.514909 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22f0ae-23b2-46a7-923f-2ac1d2b6fc9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:35.516574 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2b22f0ae-23b2-46a7-923f-2ac1d2b6fc9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:35.563907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:35.631290 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22f0ae-23b2-46a7-923f-2ac1d2b6fc9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:35.633189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22f0ae-23b2-46a7-923f-2ac1d2b6fc9e service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:35.657259 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 704/1406] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:37.646310 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc426d35-0374-4b80-bfe7-a71687e28e17 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:37.646918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc426d35-0374-4b80-bfe7-a71687e28e17 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:37.646918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc426d35-0374-4b80-bfe7-a71687e28e17 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:37.648414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 703/1407] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:37.653760 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec03a6-0e1d-4d3a-9276-74dc0041e11d None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:37.654231 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec03a6-0e1d-4d3a-9276-74dc0041e11d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:37.654341 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec03a6-0e1d-4d3a-9276-74dc0041e11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:37.733236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-e3ec03a6-0e1d-4d3a-9276-74dc0041e11d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:37.734507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e3ec03a6-0e1d-4d3a-9276-74dc0041e11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:37.886080 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 705/1408] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:38.178639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc426d35-0374-4b80-bfe7-a71687e28e17 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:38.180353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-fc426d35-0374-4b80-bfe7-a71687e28e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:38.390699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc426d35-0374-4b80-bfe7-a71687e28e17 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:38.483673 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:38.539573 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc1588-fbc0-43c4-83e2-3f38c37c3498 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:38.540242 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc1588-fbc0-43c4-83e2-3f38c37c3498 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:38.540563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc1588-fbc0-43c4-83e2-3f38c37c3498 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:38.540846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc1588-fbc0-43c4-83e2-3f38c37c3498 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:38.542689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a4cc1588-fbc0-43c4-83e2-3f38c37c3498 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:38.675394 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc1588-fbc0-43c4-83e2-3f38c37c3498 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:38.677853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc1588-fbc0-43c4-83e2-3f38c37c3498 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:38.701971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 704/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:37:47.883759 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=93 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:48.696201 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=65 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:53.445821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3d2e6-a4be-4d42-94d5-4cd5b44c6ca9 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:53.446478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3d2e6-a4be-4d42-94d5-4cd5b44c6ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:53.446478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3d2e6-a4be-4d42-94d5-4cd5b44c6ca9 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:53.447834 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 706/1410] 10.222.0.25 () {54 vars in 918 bytes} [Tue Aug 30 14:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:53.453740 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e4cbb-a88a-4396-9132-c645a0e2a6a4 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:53.454226 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e4cbb-a88a-4396-9132-c645a0e2a6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:53.454321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e4cbb-a88a-4396-9132-c645a0e2a6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:53.532897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ef2e4cbb-a88a-4396-9132-c645a0e2a6a4 None None] MFA Rules not processed for user `d7cdd6b1ee9f4e97beb9bb037e446c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:53.534122 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ef2e4cbb-a88a-4396-9132-c645a0e2a6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:53.647182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 705/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Tue Aug 30 14:37:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:53.662469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9989a-a588-4f75-b428-aa745f5fb211 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:53.662746 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9989a-a588-4f75-b428-aa745f5fb211 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:53.663047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9989a-a588-4f75-b428-aa745f5fb211 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:53.664448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 707/1412] 10.222.0.25 () {54 vars in 918 bytes} [Tue Aug 30 14:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:53.669457 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1bec3-86d9-40b5-af97-3d20099b4f57 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:53.669743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1bec3-86d9-40b5-af97-3d20099b4f57 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:53.670011 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1bec3-86d9-40b5-af97-3d20099b4f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:53.773563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:53.777731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-28f1bec3-86d9-40b5-af97-3d20099b4f57 None None] MFA Rules not processed for user `cc95bbdafb0c4f34ad0662bc2e3ba4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:53.779480 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-28f1bec3-86d9-40b5-af97-3d20099b4f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:53.907061 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 706/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Tue Aug 30 14:37:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:56.394867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba330c-c7c8-4817-95e1-19aafa152172 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:56.395442 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba330c-c7c8-4817-95e1-19aafa152172 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:56.395442 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba330c-c7c8-4817-95e1-19aafa152172 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:56.396864 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:56.402651 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0254804-015a-4c0c-aa8a-f58c0da1441f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:56.403100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0254804-015a-4c0c-aa8a-f58c0da1441f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:56.403260 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0254804-015a-4c0c-aa8a-f58c0da1441f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:56.474622 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a0254804-015a-4c0c-aa8a-f58c0da1441f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:56.475836 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a0254804-015a-4c0c-aa8a-f58c0da1441f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:56.534101 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:56.577582 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:56.864726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba330c-c7c8-4817-95e1-19aafa152172 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:56.866335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-58ba330c-c7c8-4817-95e1-19aafa152172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:57.016976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba330c-c7c8-4817-95e1-19aafa152172 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:57.094250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:57.140017 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7999d7-3a60-4a23-b0c4-d617e9817597 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:57.140659 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7999d7-3a60-4a23-b0c4-d617e9817597 service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:57.140934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7999d7-3a60-4a23-b0c4-d617e9817597 service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:57.141199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7999d7-3a60-4a23-b0c4-d617e9817597 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:57.142819 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-fd7999d7-3a60-4a23-b0c4-d617e9817597 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:57.236987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7999d7-3a60-4a23-b0c4-d617e9817597 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:57.238612 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7999d7-3a60-4a23-b0c4-d617e9817597 service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:57.257968 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 709/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:58.182914 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b044a3d-3ea3-4fd4-a837-881b5890e02a None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:58.183517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b044a3d-3ea3-4fd4-a837-881b5890e02a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:58.183517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b044a3d-3ea3-4fd4-a837-881b5890e02a None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:58.184902 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:58.192635 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6028a-2f54-41d1-a317-c3f34d57d479 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:58.193409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6028a-2f54-41d1-a317-c3f34d57d479 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:58.193587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6028a-2f54-41d1-a317-c3f34d57d479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:58.287020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-3cf6028a-2f54-41d1-a317-c3f34d57d479 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:58.288397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-3cf6028a-2f54-41d1-a317-c3f34d57d479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:58.324143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:58.402031 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:58.551322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b044a3d-3ea3-4fd4-a837-881b5890e02a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:58.552830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7b044a3d-3ea3-4fd4-a837-881b5890e02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:58.728685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b044a3d-3ea3-4fd4-a837-881b5890e02a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:58.771070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:58.848022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33391d4d-d4b8-4944-adfc-fd59ee0b006f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:58.848549 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33391d4d-d4b8-4944-adfc-fd59ee0b006f service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:58.848809 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33391d4d-d4b8-4944-adfc-fd59ee0b006f service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:58.849090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33391d4d-d4b8-4944-adfc-fd59ee0b006f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:58.850696 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-33391d4d-d4b8-4944-adfc-fd59ee0b006f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:58.949761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33391d4d-d4b8-4944-adfc-fd59ee0b006f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:58.951381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33391d4d-d4b8-4944-adfc-fd59ee0b006f service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:58.969436 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 709/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:59.849268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b96b0-f87d-4b85-b071-60018723f730 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:59.849837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b96b0-f87d-4b85-b071-60018723f730 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:59.849837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b96b0-f87d-4b85-b071-60018723f730 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:59.851207 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:59.856962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8bbc6-04ee-4ed4-90a2-5b66a4f8006d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:59.857473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8bbc6-04ee-4ed4-90a2-5b66a4f8006d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:59.857473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8bbc6-04ee-4ed4-90a2-5b66a4f8006d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:59.915006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:59.939384 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4ad8bbc6-04ee-4ed4-90a2-5b66a4f8006d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:59.940652 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4ad8bbc6-04ee-4ed4-90a2-5b66a4f8006d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:00.040078 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:00.193772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b96b0-f87d-4b85-b071-60018723f730 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:00.196457 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-436b96b0-f87d-4b85-b071-60018723f730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:00.358131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b96b0-f87d-4b85-b071-60018723f730 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:00.371331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:00.488684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da8c7e-b147-48c8-8fa1-0ada99c6b1f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:00.489204 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da8c7e-b147-48c8-8fa1-0ada99c6b1f9 service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:00.489481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da8c7e-b147-48c8-8fa1-0ada99c6b1f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:00.489743 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da8c7e-b147-48c8-8fa1-0ada99c6b1f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:00.491388 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e7da8c7e-b147-48c8-8fa1-0ada99c6b1f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:00.593444 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da8c7e-b147-48c8-8fa1-0ada99c6b1f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:00.595079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da8c7e-b147-48c8-8fa1-0ada99c6b1f9 service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:00.616220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 712/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:38:01.493840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fd30a-95ac-44b9-9ff1-bfb874c010f1 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:01.494902 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fd30a-95ac-44b9-9ff1-bfb874c010f1 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:01.494902 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fd30a-95ac-44b9-9ff1-bfb874c010f1 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:01.495857 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:01.501617 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296477b-3834-4c4b-aeca-a162f2eb7105 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:01.502154 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296477b-3834-4c4b-aeca-a162f2eb7105 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:01.502154 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296477b-3834-4c4b-aeca-a162f2eb7105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:01.524874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:01.585591 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-8296477b-3834-4c4b-aeca-a162f2eb7105 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:01.586829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-8296477b-3834-4c4b-aeca-a162f2eb7105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:01.688593 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:01.837531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595fd30a-95ac-44b9-9ff1-bfb874c010f1 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:01.839078 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-595fd30a-95ac-44b9-9ff1-bfb874c010f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:01.952880 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:01.992458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595fd30a-95ac-44b9-9ff1-bfb874c010f1 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:02.113238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be8e18-e105-47bf-9948-4c82c9fda5d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:02.113862 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be8e18-e105-47bf-9948-4c82c9fda5d4 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:02.114137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be8e18-e105-47bf-9948-4c82c9fda5d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:02.114404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be8e18-e105-47bf-9948-4c82c9fda5d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:02.116040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-25be8e18-e105-47bf-9948-4c82c9fda5d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:02.197012 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:02.212368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be8e18-e105-47bf-9948-4c82c9fda5d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:02.214618 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be8e18-e105-47bf-9948-4c82c9fda5d4 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:02.234103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 712/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:38:03.100336 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e11fab-ee13-4b27-b56e-a8a2bac55ab2 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:03.100895 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e11fab-ee13-4b27-b56e-a8a2bac55ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:03.100895 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e11fab-ee13-4b27-b56e-a8a2bac55ab2 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:03.102211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:03.108094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eccb3b-0a0b-48aa-b3b0-e4c770233762 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:03.108597 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eccb3b-0a0b-48aa-b3b0-e4c770233762 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:03.108716 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eccb3b-0a0b-48aa-b3b0-e4c770233762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:03.188533 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e9eccb3b-0a0b-48aa-b3b0-e4c770233762 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:03.189776 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e9eccb3b-0a0b-48aa-b3b0-e4c770233762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:03.300947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:03.456021 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e11fab-ee13-4b27-b56e-a8a2bac55ab2 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:03.458161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-93e11fab-ee13-4b27-b56e-a8a2bac55ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:03.551465 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:03.617497 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e11fab-ee13-4b27-b56e-a8a2bac55ab2 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:03.735111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb8ab3-49e7-4d6c-9c3f-e77e24801ed2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:03.735638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb8ab3-49e7-4d6c-9c3f-e77e24801ed2 service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:03.735947 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb8ab3-49e7-4d6c-9c3f-e77e24801ed2 service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:03.736248 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb8ab3-49e7-4d6c-9c3f-e77e24801ed2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:03.737860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-44bb8ab3-49e7-4d6c-9c3f-e77e24801ed2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:03.790389 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:03.834743 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb8ab3-49e7-4d6c-9c3f-e77e24801ed2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:03.836385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb8ab3-49e7-4d6c-9c3f-e77e24801ed2 service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:03.855919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 715/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:38:04.672042 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93465eb9-b964-4609-a62c-1e3669944711 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:04.672629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93465eb9-b964-4609-a62c-1e3669944711 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:04.672629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93465eb9-b964-4609-a62c-1e3669944711 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:04.673926 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:04.679298 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d80fb-780c-4180-95f1-c39c941de4f0 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:04.679734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d80fb-780c-4180-95f1-c39c941de4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:04.679842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d80fb-780c-4180-95f1-c39c941de4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:04.763118 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-d15d80fb-780c-4180-95f1-c39c941de4f0 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:04.765050 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d15d80fb-780c-4180-95f1-c39c941de4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:04.869277 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:05.024887 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93465eb9-b964-4609-a62c-1e3669944711 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:05.026443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-93465eb9-b964-4609-a62c-1e3669944711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:05.079816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:05.181542 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93465eb9-b964-4609-a62c-1e3669944711 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:05.301910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b028ed-62d2-47ab-ae75-9edfed4fe777 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:05.302447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b028ed-62d2-47ab-ae75-9edfed4fe777 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:05.302700 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b028ed-62d2-47ab-ae75-9edfed4fe777 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:05.302994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b028ed-62d2-47ab-ae75-9edfed4fe777 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:05.304731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-22b028ed-62d2-47ab-ae75-9edfed4fe777 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:05.312816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:05.407615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b028ed-62d2-47ab-ae75-9edfed4fe777 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:05.409279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b028ed-62d2-47ab-ae75-9edfed4fe777 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:05.428844 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 715/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:38:06.238597 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca79acc-0b51-48ff-bf32-f2f868147f62 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:06.239309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca79acc-0b51-48ff-bf32-f2f868147f62 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:06.239309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca79acc-0b51-48ff-bf32-f2f868147f62 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:06.240682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:06.246723 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a29a6-245a-4909-b24f-ca37deb05a54 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:06.247255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a29a6-245a-4909-b24f-ca37deb05a54 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:06.247255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a29a6-245a-4909-b24f-ca37deb05a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:06.331537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4f9a29a6-245a-4909-b24f-ca37deb05a54 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:06.332802 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4f9a29a6-245a-4909-b24f-ca37deb05a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:06.401973 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:06.434031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:06.593429 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca79acc-0b51-48ff-bf32-f2f868147f62 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:06.594953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-aca79acc-0b51-48ff-bf32-f2f868147f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:06.614996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:06.756225 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca79acc-0b51-48ff-bf32-f2f868147f62 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:06.865085 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:06.897915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be7b64-d0f4-491b-8eef-f04b55d1b8c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:06.898612 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be7b64-d0f4-491b-8eef-f04b55d1b8c9 service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:06.898933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be7b64-d0f4-491b-8eef-f04b55d1b8c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:06.899269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be7b64-d0f4-491b-8eef-f04b55d1b8c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:06.901372 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f4be7b64-d0f4-491b-8eef-f04b55d1b8c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:07.018164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be7b64-d0f4-491b-8eef-f04b55d1b8c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:07.019822 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be7b64-d0f4-491b-8eef-f04b55d1b8c9 service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:07.041999 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 718/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:38:07.858481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af051d8-5e15-4a84-ac66-05696ca4c61a None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:07.859072 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af051d8-5e15-4a84-ac66-05696ca4c61a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:07.859072 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af051d8-5e15-4a84-ac66-05696ca4c61a None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:07.860431 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:07.865702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba11af8-054b-43a9-b895-73d04fd5fa37 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:07.866175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba11af8-054b-43a9-b895-73d04fd5fa37 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:07.866268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba11af8-054b-43a9-b895-73d04fd5fa37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:07.947501 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-0ba11af8-054b-43a9-b895-73d04fd5fa37 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:07.948843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0ba11af8-054b-43a9-b895-73d04fd5fa37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:07.992156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:08.050517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:38:08.200664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af051d8-5e15-4a84-ac66-05696ca4c61a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:08.202173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8af051d8-5e15-4a84-ac66-05696ca4c61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:08.357174 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af051d8-5e15-4a84-ac66-05696ca4c61a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:08.412440 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:08.480380 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4de4d-bb65-476c-9e0f-bf423c436fa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:08.480909 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4de4d-bb65-476c-9e0f-bf423c436fa1 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:08.481180 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4de4d-bb65-476c-9e0f-bf423c436fa1 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:08.481441 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4de4d-bb65-476c-9e0f-bf423c436fa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:08.483080 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5ac4de4d-bb65-476c-9e0f-bf423c436fa1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:08.579922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4de4d-bb65-476c-9e0f-bf423c436fa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:08.581652 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4de4d-bb65-476c-9e0f-bf423c436fa1 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:08.601844 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 718/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:38:09.415522 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b57d2-363e-4b7c-92e1-63d8a8585878 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:09.416206 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b57d2-363e-4b7c-92e1-63d8a8585878 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:09.416206 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b57d2-363e-4b7c-92e1-63d8a8585878 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:09.417661 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:09.423279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc926e-dcd5-4ade-98aa-c66f00bea202 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:09.423833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc926e-dcd5-4ade-98aa-c66f00bea202 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:09.423833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc926e-dcd5-4ade-98aa-c66f00bea202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:09.505391 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2fcc926e-dcd5-4ade-98aa-c66f00bea202 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:09.506727 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2fcc926e-dcd5-4ade-98aa-c66f00bea202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:09.509286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:09.615455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:09.778085 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b57d2-363e-4b7c-92e1-63d8a8585878 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:09.779551 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-949b57d2-363e-4b7c-92e1-63d8a8585878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:09.967275 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b57d2-363e-4b7c-92e1-63d8a8585878 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:09.997884 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:10.102772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9624418-b6d7-40ae-a614-25398b90fd1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:10.103359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9624418-b6d7-40ae-a614-25398b90fd1a service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:10.104243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9624418-b6d7-40ae-a614-25398b90fd1a service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:10.104537 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9624418-b6d7-40ae-a614-25398b90fd1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:10.106154 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e9624418-b6d7-40ae-a614-25398b90fd1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:10.201792 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9624418-b6d7-40ae-a614-25398b90fd1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:10.203419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9624418-b6d7-40ae-a614-25398b90fd1a service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:10.223011 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 721/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:38:12.030747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f75479-62fb-4ab3-8577-48897b8197c1 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:12.031318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f75479-62fb-4ab3-8577-48897b8197c1 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:12.031318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f75479-62fb-4ab3-8577-48897b8197c1 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:12.032585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:12.038035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44483bd4-c880-4d98-b10a-b2f082aed492 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:12.038489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44483bd4-c880-4d98-b10a-b2f082aed492 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:12.038599 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44483bd4-c880-4d98-b10a-b2f082aed492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:12.083364 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:12.112285 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-44483bd4-c880-4d98-b10a-b2f082aed492 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:12.113537 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-44483bd4-c880-4d98-b10a-b2f082aed492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:12.221085 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:12.506227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f75479-62fb-4ab3-8577-48897b8197c1 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:12.507799 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-12f75479-62fb-4ab3-8577-48897b8197c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:12.632138 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f75479-62fb-4ab3-8577-48897b8197c1 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:12.672124 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:12.724057 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc208f-3812-4625-8cd3-8879f61ccc33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:12.724670 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc208f-3812-4625-8cd3-8879f61ccc33 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:12.724947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc208f-3812-4625-8cd3-8879f61ccc33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:12.725216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc208f-3812-4625-8cd3-8879f61ccc33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:12.726882 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-15cc208f-3812-4625-8cd3-8879f61ccc33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:12.847117 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc208f-3812-4625-8cd3-8879f61ccc33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:12.848620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc208f-3812-4625-8cd3-8879f61ccc33 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:12.869412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 721/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:38:14.914567 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e4ae1-0b72-4dee-8835-58a9aefe150b None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:14.915123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e4ae1-0b72-4dee-8835-58a9aefe150b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:14.915123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e4ae1-0b72-4dee-8835-58a9aefe150b None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:14.916696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:14.922558 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba7e5b-755e-41cb-90cd-ad84db03e2f9 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:14.923090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba7e5b-755e-41cb-90cd-ad84db03e2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:14.923090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba7e5b-755e-41cb-90cd-ad84db03e2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:14.999579 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2eba7e5b-755e-41cb-90cd-ad84db03e2f9 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:15.001010 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2eba7e5b-755e-41cb-90cd-ad84db03e2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:15.015695 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:15.128299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:38:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:15.407129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e4ae1-0b72-4dee-8835-58a9aefe150b None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:15.408938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-526e4ae1-0b72-4dee-8835-58a9aefe150b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:15.537383 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e4ae1-0b72-4dee-8835-58a9aefe150b None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:15.554333 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:15.624818 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd03d6-8367-4a94-a92c-456529aabbf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:15.625437 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd03d6-8367-4a94-a92c-456529aabbf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:15.625702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd03d6-8367-4a94-a92c-456529aabbf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:15.625967 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd03d6-8367-4a94-a92c-456529aabbf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:15.627576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d8cd03d6-8367-4a94-a92c-456529aabbf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:15.743506 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd03d6-8367-4a94-a92c-456529aabbf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:15.745609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd03d6-8367-4a94-a92c-456529aabbf5 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:15.764825 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 724/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:38:18.013291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badf76a-0f21-480a-9f65-cf4e3c16eb86 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:18.013905 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badf76a-0f21-480a-9f65-cf4e3c16eb86 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:18.013905 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badf76a-0f21-480a-9f65-cf4e3c16eb86 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:18.015099 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:18.020600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda37e8-78dc-4997-bec9-c4f7da6e0be8 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:18.021035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda37e8-78dc-4997-bec9-c4f7da6e0be8 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:18.021126 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda37e8-78dc-4997-bec9-c4f7da6e0be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:18.070451 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:18.090853 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-edda37e8-78dc-4997-bec9-c4f7da6e0be8 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:18.092204 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-edda37e8-78dc-4997-bec9-c4f7da6e0be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:18.195053 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:18.477480 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badf76a-0f21-480a-9f65-cf4e3c16eb86 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:18.479294 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8badf76a-0f21-480a-9f65-cf4e3c16eb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:18.602472 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badf76a-0f21-480a-9f65-cf4e3c16eb86 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:18.646119 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:18.683086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a5446-a424-4f28-91ac-5a0c5d19c6de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:18.683698 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a5446-a424-4f28-91ac-5a0c5d19c6de service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:18.683965 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a5446-a424-4f28-91ac-5a0c5d19c6de service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:18.684279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a5446-a424-4f28-91ac-5a0c5d19c6de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:18.685906 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-335a5446-a424-4f28-91ac-5a0c5d19c6de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:18.817159 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a5446-a424-4f28-91ac-5a0c5d19c6de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:18.818621 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a5446-a424-4f28-91ac-5a0c5d19c6de service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:18.839163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 724/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:38:20.898984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd6f0f-7fdc-4028-b4e8-99641a3215a4 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:20.899527 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd6f0f-7fdc-4028-b4e8-99641a3215a4 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:20.899527 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd6f0f-7fdc-4028-b4e8-99641a3215a4 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:20.901297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:38:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:20.907024 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7d90a-9500-4836-8efb-332e181daab3 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:20.907508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7d90a-9500-4836-8efb-332e181daab3 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:20.907622 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7d90a-9500-4836-8efb-332e181daab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:20.979616 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e3f7d90a-9500-4836-8efb-332e181daab3 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:20.980884 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e3f7d90a-9500-4836-8efb-332e181daab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:21.003572 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:21.087485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:21.376126 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd6f0f-7fdc-4028-b4e8-99641a3215a4 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:21.377729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-02dd6f0f-7fdc-4028-b4e8-99641a3215a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:21.490570 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd6f0f-7fdc-4028-b4e8-99641a3215a4 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:21.513224 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:21.570090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aeeb74-b227-487b-9f9e-97397e0cdd5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:21.570708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aeeb74-b227-487b-9f9e-97397e0cdd5f service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:21.571009 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aeeb74-b227-487b-9f9e-97397e0cdd5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:21.571261 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aeeb74-b227-487b-9f9e-97397e0cdd5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:21.573554 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c1aeeb74-b227-487b-9f9e-97397e0cdd5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:21.688592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aeeb74-b227-487b-9f9e-97397e0cdd5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:21.690040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aeeb74-b227-487b-9f9e-97397e0cdd5f service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:21.712140 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 727/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:38:23.748441 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99177fee-6269-4d74-853c-4e1771576ffa None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:23.749037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99177fee-6269-4d74-853c-4e1771576ffa None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:23.749037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99177fee-6269-4d74-853c-4e1771576ffa None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:23.750317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:23.755737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c15be-4a73-4858-847b-9106965f2eb6 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:23.756256 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c15be-4a73-4858-847b-9106965f2eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:23.756256 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c15be-4a73-4858-847b-9106965f2eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:23.820736 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:23.825389 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-2b8c15be-4a73-4858-847b-9106965f2eb6 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:23.827161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2b8c15be-4a73-4858-847b-9106965f2eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:23.933536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:24.223576 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99177fee-6269-4d74-853c-4e1771576ffa None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:24.225192 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-99177fee-6269-4d74-853c-4e1771576ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:24.336944 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99177fee-6269-4d74-853c-4e1771576ffa None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:24.387120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:24.415028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb3b5a-c650-4df6-a3f9-46c1b41c3ad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:24.415686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb3b5a-c650-4df6-a3f9-46c1b41c3ad6 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:24.415953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb3b5a-c650-4df6-a3f9-46c1b41c3ad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:24.416244 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb3b5a-c650-4df6-a3f9-46c1b41c3ad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:24.417870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-88cb3b5a-c650-4df6-a3f9-46c1b41c3ad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:24.537291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb3b5a-c650-4df6-a3f9-46c1b41c3ad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:24.538645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb3b5a-c650-4df6-a3f9-46c1b41c3ad6 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:24.557039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 727/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:38:26.560928 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687794f2-ae70-42fc-a672-17e7ff0d2546 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:26.561528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687794f2-ae70-42fc-a672-17e7ff0d2546 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:26.561528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687794f2-ae70-42fc-a672-17e7ff0d2546 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:26.562831 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:26.568474 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c333e85-f423-4821-be78-3da9b993de47 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:26.568936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c333e85-f423-4821-be78-3da9b993de47 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:26.569038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c333e85-f423-4821-be78-3da9b993de47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:26.642186 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1c333e85-f423-4821-be78-3da9b993de47 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:26.643416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1c333e85-f423-4821-be78-3da9b993de47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:26.675061 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:26.749843 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:38:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:27.032821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687794f2-ae70-42fc-a672-17e7ff0d2546 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:27.034424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-687794f2-ae70-42fc-a672-17e7ff0d2546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:27.143777 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687794f2-ae70-42fc-a672-17e7ff0d2546 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:27.177058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:27.226369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d9e12-ec48-49c4-8d5a-9ec5ae990c9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:27.226995 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d9e12-ec48-49c4-8d5a-9ec5ae990c9b service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:27.227275 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d9e12-ec48-49c4-8d5a-9ec5ae990c9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:27.227793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d9e12-ec48-49c4-8d5a-9ec5ae990c9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:27.229276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e28d9e12-ec48-49c4-8d5a-9ec5ae990c9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:27.348294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d9e12-ec48-49c4-8d5a-9ec5ae990c9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:27.349662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d9e12-ec48-49c4-8d5a-9ec5ae990c9b service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:27.370401 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 730/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:38:29.348190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a1499-93a8-4553-8693-e2f122905d0a None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:29.348890 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a1499-93a8-4553-8693-e2f122905d0a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:29.348890 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a1499-93a8-4553-8693-e2f122905d0a None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:29.350193 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:29.356225 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad975e7-1142-40e2-b8e1-47891d950e06 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:29.356702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad975e7-1142-40e2-b8e1-47891d950e06 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:29.356801 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad975e7-1142-40e2-b8e1-47891d950e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:29.426584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-fad975e7-1142-40e2-b8e1-47891d950e06 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:29.427810 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-fad975e7-1142-40e2-b8e1-47891d950e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:29.436715 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:29.536237 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:29.831516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a1499-93a8-4553-8693-e2f122905d0a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:29.833094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6a7a1499-93a8-4553-8693-e2f122905d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:29.939192 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a1499-93a8-4553-8693-e2f122905d0a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:30.001439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:30.022693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe5221-27f7-4ce6-bd81-00b5df1305ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:30.023305 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe5221-27f7-4ce6-bd81-00b5df1305ca service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:30.023579 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe5221-27f7-4ce6-bd81-00b5df1305ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:30.023838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe5221-27f7-4ce6-bd81-00b5df1305ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:30.025690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-afbe5221-27f7-4ce6-bd81-00b5df1305ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:30.143246 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe5221-27f7-4ce6-bd81-00b5df1305ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:30.144739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe5221-27f7-4ce6-bd81-00b5df1305ca service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:30.164845 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 730/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:38:32.117888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a86c0-da7f-4bf8-8723-1dddf690ef35 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:32.119227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a86c0-da7f-4bf8-8723-1dddf690ef35 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:32.119227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a86c0-da7f-4bf8-8723-1dddf690ef35 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:32.120069 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:32.126444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11fc77-e39f-4571-9f36-7386641b3f96 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:32.127017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11fc77-e39f-4571-9f36-7386641b3f96 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:32.127017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11fc77-e39f-4571-9f36-7386641b3f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:32.208991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8c11fc77-e39f-4571-9f36-7386641b3f96 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:32.210245 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8c11fc77-e39f-4571-9f36-7386641b3f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:32.261097 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:32.326169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:32.605105 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a86c0-da7f-4bf8-8723-1dddf690ef35 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:32.606912 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-578a86c0-da7f-4bf8-8723-1dddf690ef35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:32.725840 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a86c0-da7f-4bf8-8723-1dddf690ef35 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:32.768333 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:32.807419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f6303-0792-449c-a25c-8d0bb3479aba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:32.808065 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f6303-0792-449c-a25c-8d0bb3479aba service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:32.808344 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f6303-0792-449c-a25c-8d0bb3479aba service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:32.808611 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f6303-0792-449c-a25c-8d0bb3479aba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:32.810299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-9e7f6303-0792-449c-a25c-8d0bb3479aba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:32.933185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f6303-0792-449c-a25c-8d0bb3479aba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:32.934650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f6303-0792-449c-a25c-8d0bb3479aba service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:32.957190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 733/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:38:34.879975 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d8337-52e8-4683-a11a-b44a3118029b None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:34.880574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d8337-52e8-4683-a11a-b44a3118029b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:34.880574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d8337-52e8-4683-a11a-b44a3118029b None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:34.881954 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:34.888033 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e9397-364a-4711-9d30-c1a833979ef2 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:34.888535 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e9397-364a-4711-9d30-c1a833979ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:34.888643 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e9397-364a-4711-9d30-c1a833979ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:34.965164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-172e9397-364a-4711-9d30-c1a833979ef2 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:34.966329 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-172e9397-364a-4711-9d30-c1a833979ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:34.986532 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:35.074115 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:35.357305 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d8337-52e8-4683-a11a-b44a3118029b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:35.358858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-db9d8337-52e8-4683-a11a-b44a3118029b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:35.480041 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d8337-52e8-4683-a11a-b44a3118029b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:35.554246 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:35.563140 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3d1f8-bae9-4fdf-9ade-cc1d23771f72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:35.563790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3d1f8-bae9-4fdf-9ade-cc1d23771f72 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:35.564097 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3d1f8-bae9-4fdf-9ade-cc1d23771f72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:35.564381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3d1f8-bae9-4fdf-9ade-cc1d23771f72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:35.566055 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-25a3d1f8-bae9-4fdf-9ade-cc1d23771f72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:35.706036 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3d1f8-bae9-4fdf-9ade-cc1d23771f72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:35.707877 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3d1f8-bae9-4fdf-9ade-cc1d23771f72 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:35.729823 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 733/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:38:37.744086 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082f732-9156-4171-a420-c1092dae37a7 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:37.744689 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082f732-9156-4171-a420-c1092dae37a7 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:37.744689 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082f732-9156-4171-a420-c1092dae37a7 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:37.746160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:37.751938 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f68d87-22a0-46ac-8b4f-c774925dfe83 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:37.752426 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f68d87-22a0-46ac-8b4f-c774925dfe83 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:37.752522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f68d87-22a0-46ac-8b4f-c774925dfe83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:37.824167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d4f68d87-22a0-46ac-8b4f-c774925dfe83 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:37.825388 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d4f68d87-22a0-46ac-8b4f-c774925dfe83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:37.877538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:37.929090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:38.205212 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082f732-9156-4171-a420-c1092dae37a7 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:38.206824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a082f732-9156-4171-a420-c1092dae37a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:38.325747 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082f732-9156-4171-a420-c1092dae37a7 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:38.385278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:38.414678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46054d-79cd-4652-8322-66aee68eb5a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:38.415446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46054d-79cd-4652-8322-66aee68eb5a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:38.415729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46054d-79cd-4652-8322-66aee68eb5a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:38.416147 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46054d-79cd-4652-8322-66aee68eb5a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:38.418674 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4c46054d-79cd-4652-8322-66aee68eb5a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:38.539265 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46054d-79cd-4652-8322-66aee68eb5a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:38.541371 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46054d-79cd-4652-8322-66aee68eb5a0 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:38.561338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 736/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:38:47.927129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=22 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:48.558435 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=66 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:58.345860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9734905-44f0-45ee-b90a-28c52b82cae8 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:58.346473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9734905-44f0-45ee-b90a-28c52b82cae8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:58.346473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9734905-44f0-45ee-b90a-28c52b82cae8 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:58.347738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 735/1471] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:58.353613 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1612182-980b-48f3-9cac-3064311ee756 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:58.354141 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1612182-980b-48f3-9cac-3064311ee756 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:58.354141 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1612182-980b-48f3-9cac-3064311ee756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:58.429508 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a1612182-980b-48f3-9cac-3064311ee756 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:58.430754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a1612182-980b-48f3-9cac-3064311ee756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:58.531201 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 737/1472] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:58.824608 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418c3c7-2972-40a3-b6a4-7fddd6ebedeb None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:58.824896 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418c3c7-2972-40a3-b6a4-7fddd6ebedeb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:58.825189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418c3c7-2972-40a3-b6a4-7fddd6ebedeb None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:58.826527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 736/1473] 10.222.0.25 () {54 vars in 967 bytes} [Tue Aug 30 14:38:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:58.831927 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe4041-e15e-42a0-acd1-0c09b6cfc000 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:58.832242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe4041-e15e-42a0-acd1-0c09b6cfc000 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:58.832515 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe4041-e15e-42a0-acd1-0c09b6cfc000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:58.917735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-6fbe4041-e15e-42a0-acd1-0c09b6cfc000 None None] MFA Rules not processed for user `69de9368993f402cb6325e5022e064d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:58.918917 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6fbe4041-e15e-42a0-acd1-0c09b6cfc000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:58.942194 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:59.000216 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 738/1474] 10.222.0.25 () {58 vars in 1064 bytes} [Tue Aug 30 14:38:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:59.008033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418c3c7-2972-40a3-b6a4-7fddd6ebedeb None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:59.009629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b418c3c7-2972-40a3-b6a4-7fddd6ebedeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:59.121124 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418c3c7-2972-40a3-b6a4-7fddd6ebedeb None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:59.203599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968859b9-a346-423e-9b74-3a47b0640d85 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69de9368993f402cb6325e5022e064d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:59.204151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968859b9-a346-423e-9b74-3a47b0640d85 service glance] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:59.204426 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968859b9-a346-423e-9b74-3a47b0640d85 service glance] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:59.204689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968859b9-a346-423e-9b74-3a47b0640d85 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:59.206348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-968859b9-a346-423e-9b74-3a47b0640d85 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:59.262762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:59.320435 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968859b9-a346-423e-9b74-3a47b0640d85 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:59.322422 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968859b9-a346-423e-9b74-3a47b0640d85 service glance] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:59.341321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 737/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:39:01.358411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8511c-524e-4b10-9888-38b5220fb867 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:01.359054 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8511c-524e-4b10-9888-38b5220fb867 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:01.359054 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8511c-524e-4b10-9888-38b5220fb867 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:01.360448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 739/1476] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:01.366563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30946aa-c4b5-4b6b-8f31-b67b273b3503 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:01.367073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30946aa-c4b5-4b6b-8f31-b67b273b3503 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:01.367073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30946aa-c4b5-4b6b-8f31-b67b273b3503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:01.435202 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a30946aa-c4b5-4b6b-8f31-b67b273b3503 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:01.436478 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a30946aa-c4b5-4b6b-8f31-b67b273b3503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:01.541341 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 738/1477] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:39:01.811537 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8511c-524e-4b10-9888-38b5220fb867 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:01.813160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7ae8511c-524e-4b10-9888-38b5220fb867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:01.962869 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8511c-524e-4b10-9888-38b5220fb867 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:02.015993 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:02.065961 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89ed36-b750-4e30-9c05-a0ae985844bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:02.066585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89ed36-b750-4e30-9c05-a0ae985844bc admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:02.066842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89ed36-b750-4e30-9c05-a0ae985844bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:02.067116 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89ed36-b750-4e30-9c05-a0ae985844bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:02.067750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89ed36-b750-4e30-9c05-a0ae985844bc admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:02.069608 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: WARNING py.warnings [None req-bf89ed36-b750-4e30-9c05-a0ae985844bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:39:02.069608 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: warnings.warn(msg) Aug 30 14:39:02.069608 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: Aug 30 14:39:02.071674 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89ed36-b750-4e30-9c05-a0ae985844bc admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:02.090914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 740/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:01 2022] GET /identity/v3/projects => generated 2553 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:39:04.033789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653ead9-3ebf-4b67-b171-bd005a71d7d8 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:04.034342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653ead9-3ebf-4b67-b171-bd005a71d7d8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:04.034342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653ead9-3ebf-4b67-b171-bd005a71d7d8 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:04.035582 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 739/1479] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:04.041385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04f4db-8cc9-41fa-b32c-bae67ec27de0 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:04.042121 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04f4db-8cc9-41fa-b32c-bae67ec27de0 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:04.042121 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04f4db-8cc9-41fa-b32c-bae67ec27de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:04.112777 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-5a04f4db-8cc9-41fa-b32c-bae67ec27de0 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:04.114028 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5a04f4db-8cc9-41fa-b32c-bae67ec27de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:04.220156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 741/1480] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:39:04.517168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653ead9-3ebf-4b67-b171-bd005a71d7d8 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:04.519507 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6653ead9-3ebf-4b67-b171-bd005a71d7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:04.530275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:04.676631 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653ead9-3ebf-4b67-b171-bd005a71d7d8 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:04.768034 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:04.801949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc77277-9736-4a13-a274-40faaed18cf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:04.802557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc77277-9736-4a13-a274-40faaed18cf1 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:04.802830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc77277-9736-4a13-a274-40faaed18cf1 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:04.803096 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc77277-9736-4a13-a274-40faaed18cf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:04.804736 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bfc77277-9736-4a13-a274-40faaed18cf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:04.901355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc77277-9736-4a13-a274-40faaed18cf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:04.903055 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc77277-9736-4a13-a274-40faaed18cf1 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:04.922702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 740/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:39:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:39:07.021250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e95ff-9500-407a-8de2-06278ef960d7 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:07.021870 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e95ff-9500-407a-8de2-06278ef960d7 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:07.021870 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e95ff-9500-407a-8de2-06278ef960d7 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:07.023157 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 742/1482] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:07.029094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bf573-b260-4c43-9ad9-c910833bc635 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:07.029561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bf573-b260-4c43-9ad9-c910833bc635 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:07.029654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bf573-b260-4c43-9ad9-c910833bc635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:07.101016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7b8bf573-b260-4c43-9ad9-c910833bc635 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:07.102420 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7b8bf573-b260-4c43-9ad9-c910833bc635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:07.153498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:07.209302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 741/1483] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:39:07.496980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e95ff-9500-407a-8de2-06278ef960d7 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:07.498518 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-ac0e95ff-9500-407a-8de2-06278ef960d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:07.524352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:07.615731 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e95ff-9500-407a-8de2-06278ef960d7 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:07.693369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0abb6-7026-4bba-80e3-909c3a502906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:07.693981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0abb6-7026-4bba-80e3-909c3a502906 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:07.694257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0abb6-7026-4bba-80e3-909c3a502906 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:07.694523 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0abb6-7026-4bba-80e3-909c3a502906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:07.696667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-28e0abb6-7026-4bba-80e3-909c3a502906 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:07.903034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:07.941222 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0abb6-7026-4bba-80e3-909c3a502906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:07.942654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0abb6-7026-4bba-80e3-909c3a502906 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:07.962794 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 743/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:39:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:39:09.930342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f4d9d-d3d5-44bf-a1f0-d9d505346711 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:09.930906 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f4d9d-d3d5-44bf-a1f0-d9d505346711 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:09.930906 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f4d9d-d3d5-44bf-a1f0-d9d505346711 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:09.932198 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 742/1485] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:09.938222 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e317e-55b8-4d7e-b4ca-df3c843e824b None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:09.938706 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e317e-55b8-4d7e-b4ca-df3c843e824b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:09.938814 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e317e-55b8-4d7e-b4ca-df3c843e824b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:10.007794 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-297e317e-55b8-4d7e-b4ca-df3c843e824b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:10.009071 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-297e317e-55b8-4d7e-b4ca-df3c843e824b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:10.121783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 744/1486] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:39:10.408871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f4d9d-d3d5-44bf-a1f0-d9d505346711 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:10.410420 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-fa5f4d9d-d3d5-44bf-a1f0-d9d505346711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:10.527216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f4d9d-d3d5-44bf-a1f0-d9d505346711 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:10.603886 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:10.606127 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ef122-596e-41ff-8917-1689b1c7525e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:10.606865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ef122-596e-41ff-8917-1689b1c7525e service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:10.607148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ef122-596e-41ff-8917-1689b1c7525e service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:10.607412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ef122-596e-41ff-8917-1689b1c7525e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:10.609868 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ee1ef122-596e-41ff-8917-1689b1c7525e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:10.728627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ef122-596e-41ff-8917-1689b1c7525e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:10.730030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ef122-596e-41ff-8917-1689b1c7525e service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:10.748784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 743/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:39:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:39:12.839954 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c724cd-42c0-4ec6-a8f3-9c3e40d3b0e2 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:12.840523 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c724cd-42c0-4ec6-a8f3-9c3e40d3b0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:12.840523 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c724cd-42c0-4ec6-a8f3-9c3e40d3b0e2 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:12.841888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 745/1488] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:12.847861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc087db7-f1e0-4df6-88b5-d31856cb051d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:12.848396 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc087db7-f1e0-4df6-88b5-d31856cb051d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:12.848396 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc087db7-f1e0-4df6-88b5-d31856cb051d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:12.923728 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-cc087db7-f1e0-4df6-88b5-d31856cb051d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:12.925001 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cc087db7-f1e0-4df6-88b5-d31856cb051d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:12.984096 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:13.030537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 744/1489] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:39:13.316938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c724cd-42c0-4ec6-a8f3-9c3e40d3b0e2 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:13.318489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-21c724cd-42c0-4ec6-a8f3-9c3e40d3b0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:13.357241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:13.432893 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c724cd-42c0-4ec6-a8f3-9c3e40d3b0e2 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:13.527733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4a2a5-9674-4dd7-a3fe-3f22b45aa050 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:13.528258 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4a2a5-9674-4dd7-a3fe-3f22b45aa050 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:13.528548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4a2a5-9674-4dd7-a3fe-3f22b45aa050 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:13.528813 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4a2a5-9674-4dd7-a3fe-3f22b45aa050 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:13.530449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-dcd4a2a5-9674-4dd7-a3fe-3f22b45aa050 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:13.614475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:13.646085 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4a2a5-9674-4dd7-a3fe-3f22b45aa050 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:13.647542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4a2a5-9674-4dd7-a3fe-3f22b45aa050 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:13.669372 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 746/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:39:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:39:15.804377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab65d7-0473-4db1-a72f-a84326b65bf0 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:15.805573 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab65d7-0473-4db1-a72f-a84326b65bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:15.805573 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab65d7-0473-4db1-a72f-a84326b65bf0 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:15.806306 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 745/1491] 10.222.0.25 () {54 vars in 927 bytes} [Tue Aug 30 14:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:15.812998 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f5270-c28c-44ba-8b3b-81706aa57f19 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:15.813568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f5270-c28c-44ba-8b3b-81706aa57f19 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:15.813568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f5270-c28c-44ba-8b3b-81706aa57f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:15.887652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-8d2f5270-c28c-44ba-8b3b-81706aa57f19 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:15.889274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-8d2f5270-c28c-44ba-8b3b-81706aa57f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:15.994611 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 747/1492] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:39:16.300063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab65d7-0473-4db1-a72f-a84326b65bf0 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:16.302143 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-56ab65d7-0473-4db1-a72f-a84326b65bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:16.413149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab65d7-0473-4db1-a72f-a84326b65bf0 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:16.492883 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29afe9-fc33-4e07-b1fc-39f603d63ae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:16.493480 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29afe9-fc33-4e07-b1fc-39f603d63ae3 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:16.493752 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29afe9-fc33-4e07-b1fc-39f603d63ae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:16.494041 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29afe9-fc33-4e07-b1fc-39f603d63ae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:16.495700 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6b29afe9-fc33-4e07-b1fc-39f603d63ae3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:16.506104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:16.630539 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29afe9-fc33-4e07-b1fc-39f603d63ae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:16.631740 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29afe9-fc33-4e07-b1fc-39f603d63ae3 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:16.652201 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 746/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:39:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:39:25.993061 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=82 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:26.649367 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=50 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:29.788119 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41a6fe-d8b6-4967-a6ac-0f19b70e2e46 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:29.788773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41a6fe-d8b6-4967-a6ac-0f19b70e2e46 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:29.788773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41a6fe-d8b6-4967-a6ac-0f19b70e2e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:29.871943 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-fe41a6fe-d8b6-4967-a6ac-0f19b70e2e46 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:29.873211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-fe41a6fe-d8b6-4967-a6ac-0f19b70e2e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:29.980769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 748/1494] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:40:29.989408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0922f-5383-4e92-9041-3132a7912e2a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:29.989939 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0922f-5383-4e92-9041-3132a7912e2a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:29.989939 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0922f-5383-4e92-9041-3132a7912e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:30.068152 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a9b0922f-5383-4e92-9041-3132a7912e2a None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:30.069775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a9b0922f-5383-4e92-9041-3132a7912e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:30.173430 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 747/1495] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:40:30.179016 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41a6fe-d8b6-4967-a6ac-0f19b70e2e46 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:30.180858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-fe41a6fe-d8b6-4967-a6ac-0f19b70e2e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:30.198395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41a6fe-d8b6-4967-a6ac-0f19b70e2e46 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:30.268660 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:30.306619 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f9a7f-bc99-41f6-b6f3-b79a79786ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:30.307242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f9a7f-bc99-41f6-b6f3-b79a79786ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:30.307513 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f9a7f-bc99-41f6-b6f3-b79a79786ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:30.307775 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f9a7f-bc99-41f6-b6f3-b79a79786ee7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:30.308467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f9a7f-bc99-41f6-b6f3-b79a79786ee7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:40:30.308766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f9a7f-bc99-41f6-b6f3-b79a79786ee7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:30.309785 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: WARNING py.warnings [None req-518f9a7f-bc99-41f6-b6f3-b79a79786ee7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:40:30.309785 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: warnings.warn(msg) Aug 30 14:40:30.309785 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: Aug 30 14:40:30.310681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f9a7f-bc99-41f6-b6f3-b79a79786ee7 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:30.323049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 749/1496] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:40:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:30.335666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0922f-5383-4e92-9041-3132a7912e2a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:30.345218 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0922f-5383-4e92-9041-3132a7912e2a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:30.410802 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:30.455617 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f5bf9-1831-4ee0-b076-624f2d08cc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:30.456279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f5bf9-1831-4ee0-b076-624f2d08cc98 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:30.456677 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f5bf9-1831-4ee0-b076-624f2d08cc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:30.456850 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f5bf9-1831-4ee0-b076-624f2d08cc98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:30.457387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f5bf9-1831-4ee0-b076-624f2d08cc98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:30.459345 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: WARNING py.warnings [None req-5d8f5bf9-1831-4ee0-b076-624f2d08cc98 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:40:30.459345 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: warnings.warn(msg) Aug 30 14:40:30.459345 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: Aug 30 14:40:30.460676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f5bf9-1831-4ee0-b076-624f2d08cc98 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:30.532027 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 748/1497] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:40:30 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:40:30.541051 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f9a7f-bc99-41f6-b6f3-b79a79786ee7 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:30.544472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-518f9a7f-bc99-41f6-b6f3-b79a79786ee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:30.587397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f9a7f-bc99-41f6-b6f3-b79a79786ee7 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:30.685876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9add96f-e2aa-4b8c-a74c-f1911f212623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:30.686480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9add96f-e2aa-4b8c-a74c-f1911f212623 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:30.686755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9add96f-e2aa-4b8c-a74c-f1911f212623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:30.687015 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9add96f-e2aa-4b8c-a74c-f1911f212623 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:30.687646 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9add96f-e2aa-4b8c-a74c-f1911f212623 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:30.688729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: WARNING py.warnings [None req-b9add96f-e2aa-4b8c-a74c-f1911f212623 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:40:30.688729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: warnings.warn(msg) Aug 30 14:40:30.688729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: Aug 30 14:40:30.689924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9add96f-e2aa-4b8c-a74c-f1911f212623 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:30.737957 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 750/1498] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:40:30 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:40:30.742564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f5bf9-1831-4ee0-b076-624f2d08cc98 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:30.744186 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5d8f5bf9-1831-4ee0-b076-624f2d08cc98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:30.810801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f5bf9-1831-4ee0-b076-624f2d08cc98 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:30.910889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:30.924389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd16b8-0a81-4278-8ba2-d65ff6b4f9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:30.925197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd16b8-0a81-4278-8ba2-d65ff6b4f9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:30.925589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd16b8-0a81-4278-8ba2-d65ff6b4f9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:30.925983 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd16b8-0a81-4278-8ba2-d65ff6b4f9b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:30.926922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd16b8-0a81-4278-8ba2-d65ff6b4f9b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:30.928351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: WARNING py.warnings [None req-9fdd16b8-0a81-4278-8ba2-d65ff6b4f9b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:40:30.928351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: warnings.warn(msg) Aug 30 14:40:30.928351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: Aug 30 14:40:30.929229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd16b8-0a81-4278-8ba2-d65ff6b4f9b4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:30.942378 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 749/1499] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:40:30.947139 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9add96f-e2aa-4b8c-a74c-f1911f212623 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:30.948712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b9add96f-e2aa-4b8c-a74c-f1911f212623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:30.960763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9add96f-e2aa-4b8c-a74c-f1911f212623 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:30.998773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:31.078211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb3435-6345-4805-b7a9-749960da25f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:31.078739 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb3435-6345-4805-b7a9-749960da25f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:31.079010 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb3435-6345-4805-b7a9-749960da25f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:31.079272 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb3435-6345-4805-b7a9-749960da25f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:31.079991 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb3435-6345-4805-b7a9-749960da25f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:31.080971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: WARNING py.warnings [None req-6cfb3435-6345-4805-b7a9-749960da25f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:40:31.080971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: warnings.warn(msg) Aug 30 14:40:31.080971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: Aug 30 14:40:31.081851 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb3435-6345-4805-b7a9-749960da25f2 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:31.095211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 751/1500] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:40:31.100386 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd16b8-0a81-4278-8ba2-d65ff6b4f9b4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:31.101930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9fdd16b8-0a81-4278-8ba2-d65ff6b4f9b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:31.145037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd16b8-0a81-4278-8ba2-d65ff6b4f9b4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:31.246692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97215d-1d0c-44f0-9dfa-e690396a906e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:31.247402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97215d-1d0c-44f0-9dfa-e690396a906e admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:31.247667 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97215d-1d0c-44f0-9dfa-e690396a906e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:31.247925 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97215d-1d0c-44f0-9dfa-e690396a906e admin admin] PATH_INFO: `/v3/projects/b192ecab655b43b6bb46ae8ba883116e/users/621f4cc78f8e4dd980ca1a47f66603f0/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:31.271254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97215d-1d0c-44f0-9dfa-e690396a906e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b192ecab655b43b6bb46ae8ba883116e, user_id=621f4cc78f8e4dd980ca1a47f66603f0, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:31.272364 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: WARNING py.warnings [None req-cb97215d-1d0c-44f0-9dfa-e690396a906e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:40:31.272364 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: warnings.warn(msg) Aug 30 14:40:31.272364 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: Aug 30 14:40:31.274314 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97215d-1d0c-44f0-9dfa-e690396a906e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:31.303646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 750/1501] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:40:31 2022] PUT /identity/v3/projects/b192ecab655b43b6bb46ae8ba883116e/users/621f4cc78f8e4dd980ca1a47f66603f0/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:40:31.316122 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7fad3-8da9-408a-b4e0-2df4472cf96c None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:31.316404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7fad3-8da9-408a-b4e0-2df4472cf96c None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:31.316671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7fad3-8da9-408a-b4e0-2df4472cf96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:31.381905 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-e6c7fad3-8da9-408a-b4e0-2df4472cf96c None None] MFA Rules not processed for user `621f4cc78f8e4dd980ca1a47f66603f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:31.383151 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e6c7fad3-8da9-408a-b4e0-2df4472cf96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:31.469066 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 752/1502] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:40:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:40:31.482969 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef8670-91c9-4a21-9b92-fd8dfeb164ef None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:31.483254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef8670-91c9-4a21-9b92-fd8dfeb164ef None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:31.483553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef8670-91c9-4a21-9b92-fd8dfeb164ef None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:31.484952 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 751/1503] 10.222.0.25 () {54 vars in 964 bytes} [Tue Aug 30 14:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:31.490847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd0023-720d-4d40-8773-dfce9e752a8b None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:31.491180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd0023-720d-4d40-8773-dfce9e752a8b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:31.491472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd0023-720d-4d40-8773-dfce9e752a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:31.524236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:31.572154 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-c2bd0023-720d-4d40-8773-dfce9e752a8b None None] MFA Rules not processed for user `2788fece401a4a7ebdb516afe2dd8ec8`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:31.573385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c2bd0023-720d-4d40-8773-dfce9e752a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:31.653766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 753/1504] 10.222.0.25 () {58 vars in 1061 bytes} [Tue Aug 30 14:40:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:40:31.664517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef8670-91c9-4a21-9b92-fd8dfeb164ef None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:31.666860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c0ef8670-91c9-4a21-9b92-fd8dfeb164ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:31.731286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:31.790268 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef8670-91c9-4a21-9b92-fd8dfeb164ef None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:31.871729 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c31e27-6cf9-4ebc-a162-17f47c44905e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:31.872372 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c31e27-6cf9-4ebc-a162-17f47c44905e service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:31.872644 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c31e27-6cf9-4ebc-a162-17f47c44905e service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:31.872903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c31e27-6cf9-4ebc-a162-17f47c44905e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:31.874536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-30c31e27-6cf9-4ebc-a162-17f47c44905e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:31.976753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c31e27-6cf9-4ebc-a162-17f47c44905e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:31.978107 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c31e27-6cf9-4ebc-a162-17f47c44905e service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:31.981669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:32.000362 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 752/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:40:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:40:32.047198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca2e5f-59c0-4b91-93d7-06463432065f None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:32.047669 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca2e5f-59c0-4b91-93d7-06463432065f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:32.047669 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca2e5f-59c0-4b91-93d7-06463432065f None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:32.049060 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 754/1506] 10.222.0.25 () {52 vars in 830 bytes} [Tue Aug 30 14:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:32.053051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c31e27-6cf9-4ebc-a162-17f47c44905e service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:32.054317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-30c31e27-6cf9-4ebc-a162-17f47c44905e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:32.171542 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c31e27-6cf9-4ebc-a162-17f47c44905e service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:32.286933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f2e29-db69-4d88-b48c-be67658680f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:32.287528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f2e29-db69-4d88-b48c-be67658680f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:32.287801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f2e29-db69-4d88-b48c-be67658680f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:32.288105 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f2e29-db69-4d88-b48c-be67658680f0 admin admin] PATH_INFO: `/v3/users/621f4cc78f8e4dd980ca1a47f66603f0` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:32.301148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f2e29-db69-4d88-b48c-be67658680f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=621f4cc78f8e4dd980ca1a47f66603f0)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:32.302191 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: WARNING py.warnings [None req-d64f2e29-db69-4d88-b48c-be67658680f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:40:32.302191 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: warnings.warn(msg) Aug 30 14:40:32.302191 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: Aug 30 14:40:32.303441 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f2e29-db69-4d88-b48c-be67658680f0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:32.313589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:32.374314 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d64f2e29-db69-4d88-b48c-be67658680f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '621f4cc78f8e4dd980ca1a47f66603f0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:40:32.389095 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d64f2e29-db69-4d88-b48c-be67658680f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '621f4cc78f8e4dd980ca1a47f66603f0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:40:32.399409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d64f2e29-db69-4d88-b48c-be67658680f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '621f4cc78f8e4dd980ca1a47f66603f0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:40:32.399898 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d64f2e29-db69-4d88-b48c-be67658680f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '621f4cc78f8e4dd980ca1a47f66603f0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:40:32.425986 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d64f2e29-db69-4d88-b48c-be67658680f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '621f4cc78f8e4dd980ca1a47f66603f0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:40:32.433253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 753/1507] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:40:32 2022] DELETE /identity/v3/users/621f4cc78f8e4dd980ca1a47f66603f0 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:40:32.437911 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfcd87-0e40-48b7-abac-2496af65d797 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:32.438240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfcd87-0e40-48b7-abac-2496af65d797 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:32.438455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfcd87-0e40-48b7-abac-2496af65d797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:32.503530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-92cfcd87-0e40-48b7-abac-2496af65d797 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:32.509052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-92cfcd87-0e40-48b7-abac-2496af65d797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:32.590632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:32.608854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 755/1508] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:40:32.620725 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f2e29-db69-4d88-b48c-be67658680f0 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:32.622111 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d64f2e29-db69-4d88-b48c-be67658680f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:32.730992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f2e29-db69-4d88-b48c-be67658680f0 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:32.792089 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:32.814642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f585799-9f97-40c3-8695-1d305cb6fbbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:32.815262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f585799-9f97-40c3-8695-1d305cb6fbbb service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:32.815535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f585799-9f97-40c3-8695-1d305cb6fbbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:32.815798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f585799-9f97-40c3-8695-1d305cb6fbbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:32.817454 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7f585799-9f97-40c3-8695-1d305cb6fbbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:32.931392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f585799-9f97-40c3-8695-1d305cb6fbbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:32.932949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f585799-9f97-40c3-8695-1d305cb6fbbb service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:33.079488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 754/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:40:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:40:33.433185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfcd87-0e40-48b7-abac-2496af65d797 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:33.434436 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-92cfcd87-0e40-48b7-abac-2496af65d797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:33.446414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfcd87-0e40-48b7-abac-2496af65d797 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:33.554815 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e8b38-3270-4aa2-9aed-f717c6cfc388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:33.555464 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e8b38-3270-4aa2-9aed-f717c6cfc388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:33.555738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e8b38-3270-4aa2-9aed-f717c6cfc388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:33.556041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e8b38-3270-4aa2-9aed-f717c6cfc388 admin admin] PATH_INFO: `/v3/projects/b192ecab655b43b6bb46ae8ba883116e` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:33.567258 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e8b38-3270-4aa2-9aed-f717c6cfc388 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b192ecab655b43b6bb46ae8ba883116e)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:33.568825 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: WARNING py.warnings [None req-1b8e8b38-3270-4aa2-9aed-f717c6cfc388 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:40:33.568825 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: warnings.warn(msg) Aug 30 14:40:33.568825 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: Aug 30 14:40:33.570067 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e8b38-3270-4aa2-9aed-f717c6cfc388 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:33.635704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-1b8e8b38-3270-4aa2-9aed-f717c6cfc388 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b192ecab655b43b6bb46ae8ba883116e', 'request_id': 'req-1b8e8b38-3270-4aa2-9aed-f717c6cfc388'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:40:33.641841 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-1b8e8b38-3270-4aa2-9aed-f717c6cfc388 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b192ecab655b43b6bb46ae8ba883116e', 'request_id': 'req-1b8e8b38-3270-4aa2-9aed-f717c6cfc388'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:40:33.655652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-1b8e8b38-3270-4aa2-9aed-f717c6cfc388 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b192ecab655b43b6bb46ae8ba883116e', 'request_id': 'req-1b8e8b38-3270-4aa2-9aed-f717c6cfc388'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:40:33.694449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-1b8e8b38-3270-4aa2-9aed-f717c6cfc388 admin admin] The token cache is being invalidate because project b192ecab655b43b6bb46ae8ba883116e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:40:33.694711 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-1b8e8b38-3270-4aa2-9aed-f717c6cfc388 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:40:33.695237 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-1b8e8b38-3270-4aa2-9aed-f717c6cfc388 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:40:33.696618 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 756/1510] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:40:33 2022] DELETE /identity/v3/projects/b192ecab655b43b6bb46ae8ba883116e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:40:43.076878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=62 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:43.691349 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:01.450389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8244f6-03dc-4547-a80f-5e62208c0cee None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:01.451020 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8244f6-03dc-4547-a80f-5e62208c0cee None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:01.451020 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8244f6-03dc-4547-a80f-5e62208c0cee None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:01.452300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:01.457780 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac0c2d-6e6b-4756-bcb1-b2ce6eb4663d None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:01.458249 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac0c2d-6e6b-4756-bcb1-b2ce6eb4663d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:01.458349 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac0c2d-6e6b-4756-bcb1-b2ce6eb4663d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:01.549925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-b4ac0c2d-6e6b-4756-bcb1-b2ce6eb4663d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:01.551397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b4ac0c2d-6e6b-4756-bcb1-b2ce6eb4663d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:01.664869 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:01.825114 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8244f6-03dc-4547-a80f-5e62208c0cee None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:01.827599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0e8244f6-03dc-4547-a80f-5e62208c0cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:01.966447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8244f6-03dc-4547-a80f-5e62208c0cee None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:01.972448 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95b9df-87cd-4176-85de-a4659741ad24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:01.973065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95b9df-87cd-4176-85de-a4659741ad24 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:01.973329 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95b9df-87cd-4176-85de-a4659741ad24 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:01.973593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95b9df-87cd-4176-85de-a4659741ad24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:01.982126 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95b9df-87cd-4176-85de-a4659741ad24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:01.983763 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95b9df-87cd-4176-85de-a4659741ad24 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:01.994386 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 756/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:02.838949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e37dc5-a0ac-463e-b355-c44942f88f84 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:02.839517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e37dc5-a0ac-463e-b355-c44942f88f84 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:02.839517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e37dc5-a0ac-463e-b355-c44942f88f84 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:02.840936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:42:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:02.846802 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5ceb8-9223-4710-a02c-43bc5b7acef7 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:02.847464 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5ceb8-9223-4710-a02c-43bc5b7acef7 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:02.847707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5ceb8-9223-4710-a02c-43bc5b7acef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:02.870290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8fe5ceb8-9223-4710-a02c-43bc5b7acef7 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:02.871667 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8fe5ceb8-9223-4710-a02c-43bc5b7acef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:02.880985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:42:03.034807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e37dc5-a0ac-463e-b355-c44942f88f84 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:03.051893 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e37dc5-a0ac-463e-b355-c44942f88f84 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:03.057890 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c1306-1e6b-4dc2-b336-6de5e12d2e28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:03.058500 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c1306-1e6b-4dc2-b336-6de5e12d2e28 service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:03.058773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c1306-1e6b-4dc2-b336-6de5e12d2e28 service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:03.059036 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c1306-1e6b-4dc2-b336-6de5e12d2e28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:03.070394 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c1306-1e6b-4dc2-b336-6de5e12d2e28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:03.072229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c1306-1e6b-4dc2-b336-6de5e12d2e28 service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:03.082870 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 759/1516] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:03.318466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee024e5b-0134-4be7-9b3e-2394fbd412ff None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:03.318730 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee024e5b-0134-4be7-9b3e-2394fbd412ff None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:03.318998 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee024e5b-0134-4be7-9b3e-2394fbd412ff None None] PATH_INFO: `/v3` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:03.320455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 758/1517] 10.222.0.57 () {54 vars in 925 bytes} [Tue Aug 30 14:42:03 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:03.327284 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a76da7-a163-4b4c-8848-8af1758e3431 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:03.327563 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a76da7-a163-4b4c-8848-8af1758e3431 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:03.327837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a76da7-a163-4b4c-8848-8af1758e3431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:03.390849 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-24a76da7-a163-4b4c-8848-8af1758e3431 None None] MFA Rules not processed for user `d7cdd6b1ee9f4e97beb9bb037e446c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:03.392097 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-24a76da7-a163-4b4c-8848-8af1758e3431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:03.409646 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:03.470780 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 760/1518] 10.222.0.57 () {58 vars in 1015 bytes} [Tue Aug 30 14:42:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:12.350738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3933779-b76a-491f-94ce-eeee79c1c1ec None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.351325 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3933779-b76a-491f-94ce-eeee79c1c1ec None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.351325 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3933779-b76a-491f-94ce-eeee79c1c1ec None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.352623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 759/1519] 10.222.0.25 () {54 vars in 961 bytes} [Tue Aug 30 14:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:12.358778 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7430ea-85ff-421b-8900-5e0a878bfa28 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.359380 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7430ea-85ff-421b-8900-5e0a878bfa28 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.359380 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7430ea-85ff-421b-8900-5e0a878bfa28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.382350 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-bc7430ea-85ff-421b-8900-5e0a878bfa28 None None] MFA Rules not processed for user `d7cdd6b1ee9f4e97beb9bb037e446c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:12.383575 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-bc7430ea-85ff-421b-8900-5e0a878bfa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:12.393901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 761/1520] 10.222.0.25 () {58 vars in 1058 bytes} [Tue Aug 30 14:42:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:12.401939 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3933779-b76a-491f-94ce-eeee79c1c1ec None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:12.417742 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3933779-b76a-491f-94ce-eeee79c1c1ec None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:12.423859 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a8589-bf13-4781-961a-5393111cd142 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cdd6b1ee9f4e97beb9bb037e446c8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:12.424511 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a8589-bf13-4781-961a-5393111cd142 service placement] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.424869 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a8589-bf13-4781-961a-5393111cd142 service placement] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.425065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a8589-bf13-4781-961a-5393111cd142 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.434757 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a8589-bf13-4781-961a-5393111cd142 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:12.436383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a8589-bf13-4781-961a-5393111cd142 service placement] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:12.446963 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 760/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:14.333807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f3b58-007e-4530-940d-311acc252281 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:14.334409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f3b58-007e-4530-940d-311acc252281 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:14.334409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f3b58-007e-4530-940d-311acc252281 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:14.335737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:14.341081 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5253f9-c7ff-41eb-9cc6-0cc12e3f5871 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:14.341533 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5253f9-c7ff-41eb-9cc6-0cc12e3f5871 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:14.341627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5253f9-c7ff-41eb-9cc6-0cc12e3f5871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:14.362837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-da5253f9-c7ff-41eb-9cc6-0cc12e3f5871 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:14.364070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-da5253f9-c7ff-41eb-9cc6-0cc12e3f5871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:14.373152 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:14.529934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f3b58-007e-4530-940d-311acc252281 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:14.545646 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f3b58-007e-4530-940d-311acc252281 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:14.551460 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ad6d4-dd07-4804-a02e-bdb2b627847c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:14.552086 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ad6d4-dd07-4804-a02e-bdb2b627847c service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:14.552393 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ad6d4-dd07-4804-a02e-bdb2b627847c service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:14.552658 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ad6d4-dd07-4804-a02e-bdb2b627847c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:14.561203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ad6d4-dd07-4804-a02e-bdb2b627847c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:14.562786 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ad6d4-dd07-4804-a02e-bdb2b627847c service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:14.573321 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 763/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:15.388899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c179b57-4f74-4bba-aee5-cf484989f202 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:15.389776 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c179b57-4f74-4bba-aee5-cf484989f202 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:15.389776 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c179b57-4f74-4bba-aee5-cf484989f202 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:15.391353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:42:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:15.396684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe353756-0541-4be5-adfa-a450fd7d0597 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:15.397141 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe353756-0541-4be5-adfa-a450fd7d0597 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:15.397234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe353756-0541-4be5-adfa-a450fd7d0597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:15.423814 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-fe353756-0541-4be5-adfa-a450fd7d0597 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:15.425080 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-fe353756-0541-4be5-adfa-a450fd7d0597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:15.435115 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:15.592966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c179b57-4f74-4bba-aee5-cf484989f202 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:15.608813 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c179b57-4f74-4bba-aee5-cf484989f202 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:15.614611 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097e942-eec5-463c-8029-463327d9a335 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:15.615807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097e942-eec5-463c-8029-463327d9a335 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:15.616113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097e942-eec5-463c-8029-463327d9a335 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:15.616383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097e942-eec5-463c-8029-463327d9a335 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:15.625268 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097e942-eec5-463c-8029-463327d9a335 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:15.626874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097e942-eec5-463c-8029-463327d9a335 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:15.638763 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 763/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:19.662327 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff97844-c141-46d2-8640-c3f2581a1ac7 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:19.662907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff97844-c141-46d2-8640-c3f2581a1ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:19.662907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff97844-c141-46d2-8640-c3f2581a1ac7 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:19.664387 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:19.669752 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e677a5-7270-46d8-9aaa-ff5f48c0be00 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:19.670206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e677a5-7270-46d8-9aaa-ff5f48c0be00 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:19.670308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e677a5-7270-46d8-9aaa-ff5f48c0be00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:19.694707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-15e677a5-7270-46d8-9aaa-ff5f48c0be00 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:19.695974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-15e677a5-7270-46d8-9aaa-ff5f48c0be00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:19.706219 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:19.993526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff97844-c141-46d2-8640-c3f2581a1ac7 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:19.996720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5ff97844-c141-46d2-8640-c3f2581a1ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:20.088238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff97844-c141-46d2-8640-c3f2581a1ac7 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:20.092421 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09498778-9108-4cdc-9b58-49a3c14b1e1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:20.093031 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09498778-9108-4cdc-9b58-49a3c14b1e1b service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:20.093302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09498778-9108-4cdc-9b58-49a3c14b1e1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:20.093577 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09498778-9108-4cdc-9b58-49a3c14b1e1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:20.103925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09498778-9108-4cdc-9b58-49a3c14b1e1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:20.106088 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09498778-9108-4cdc-9b58-49a3c14b1e1b service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:20.119654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 766/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:22.060880 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c4563-c0d4-4152-a108-aa970a65de92 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:22.061440 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c4563-c0d4-4152-a108-aa970a65de92 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:22.061440 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c4563-c0d4-4152-a108-aa970a65de92 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:22.062686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:22.069159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c12220-4c30-40b8-858f-b650224e533f None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:22.069628 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c12220-4c30-40b8-858f-b650224e533f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:22.069723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c12220-4c30-40b8-858f-b650224e533f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:22.092294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-b0c12220-4c30-40b8-858f-b650224e533f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:22.093536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b0c12220-4c30-40b8-858f-b650224e533f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:22.104499 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:22.389232 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c4563-c0d4-4152-a108-aa970a65de92 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:22.401945 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c4563-c0d4-4152-a108-aa970a65de92 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:22.406100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff606a-a18e-4703-8dcb-4432b7329f38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:22.406681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff606a-a18e-4703-8dcb-4432b7329f38 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:22.406955 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff606a-a18e-4703-8dcb-4432b7329f38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:22.407222 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff606a-a18e-4703-8dcb-4432b7329f38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:22.416896 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff606a-a18e-4703-8dcb-4432b7329f38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:22.418662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff606a-a18e-4703-8dcb-4432b7329f38 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:22.428657 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 766/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:25.952641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546ec5a-98e9-429b-b446-8d8a4bffda27 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:25.953487 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546ec5a-98e9-429b-b446-8d8a4bffda27 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:25.953487 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546ec5a-98e9-429b-b446-8d8a4bffda27 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:25.955481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:42:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:25.960980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8072f0-c4a6-4bc3-bd53-98fb9d162983 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:25.961433 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8072f0-c4a6-4bc3-bd53-98fb9d162983 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:25.961433 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8072f0-c4a6-4bc3-bd53-98fb9d162983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:25.969937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=81 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:25.982966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8b8072f0-c4a6-4bc3-bd53-98fb9d162983 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:25.984204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8b8072f0-c4a6-4bc3-bd53-98fb9d162983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:25.992834 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:42:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:26.545163 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546ec5a-98e9-429b-b446-8d8a4bffda27 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:26.547081 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5546ec5a-98e9-429b-b446-8d8a4bffda27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:26.622488 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:26.636799 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546ec5a-98e9-429b-b446-8d8a4bffda27 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:26.640509 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504dc93-6087-4a25-bc2d-2552fa1f4504 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:26.641111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504dc93-6087-4a25-bc2d-2552fa1f4504 service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:26.641383 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504dc93-6087-4a25-bc2d-2552fa1f4504 service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:26.641647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504dc93-6087-4a25-bc2d-2552fa1f4504 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:26.651516 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504dc93-6087-4a25-bc2d-2552fa1f4504 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:26.652997 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504dc93-6087-4a25-bc2d-2552fa1f4504 service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:26.662748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 769/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:28.588165 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18ce7d-9f9f-47f3-87f0-1c69ba7a010f None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:28.588971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18ce7d-9f9f-47f3-87f0-1c69ba7a010f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:28.588971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18ce7d-9f9f-47f3-87f0-1c69ba7a010f None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:28.591008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:42:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:28.597379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee08503-f9c0-4029-8ba2-c17aa1bbac53 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:28.598121 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee08503-f9c0-4029-8ba2-c17aa1bbac53 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:28.598121 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee08503-f9c0-4029-8ba2-c17aa1bbac53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:28.625512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-8ee08503-f9c0-4029-8ba2-c17aa1bbac53 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:28.626844 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-8ee08503-f9c0-4029-8ba2-c17aa1bbac53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:28.639065 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:29.187654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18ce7d-9f9f-47f3-87f0-1c69ba7a010f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:29.189711 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-fc18ce7d-9f9f-47f3-87f0-1c69ba7a010f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:29.205251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18ce7d-9f9f-47f3-87f0-1c69ba7a010f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:29.209237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc463135-5033-4ffe-a64d-203d54c7d4fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:29.209834 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc463135-5033-4ffe-a64d-203d54c7d4fd service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:29.210102 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc463135-5033-4ffe-a64d-203d54c7d4fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:29.210364 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc463135-5033-4ffe-a64d-203d54c7d4fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:29.220503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc463135-5033-4ffe-a64d-203d54c7d4fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:29.221943 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc463135-5033-4ffe-a64d-203d54c7d4fd service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:29.232481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 769/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:38.620275 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=11 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:39.227747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=8 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:55.981919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a46e3-98df-4ed1-a8bb-af4790a92aaf None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:55.984514 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a46e3-98df-4ed1-a8bb-af4790a92aaf None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:55.985341 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a46e3-98df-4ed1-a8bb-af4790a92aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.031937 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-f47a46e3-98df-4ed1-a8bb-af4790a92aaf None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:56.035656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f47a46e3-98df-4ed1-a8bb-af4790a92aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:56.047415 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 771/1540] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:56.056727 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3a13f-ef54-4d02-97e6-49be67959325 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:56.057674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3a13f-ef54-4d02-97e6-49be67959325 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:56.057674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3a13f-ef54-4d02-97e6-49be67959325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.089345 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a8e3a13f-ef54-4d02-97e6-49be67959325 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:56.090617 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a8e3a13f-ef54-4d02-97e6-49be67959325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:56.103250 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 770/1541] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:56.108710 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a46e3-98df-4ed1-a8bb-af4790a92aaf None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:56.135876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a46e3-98df-4ed1-a8bb-af4790a92aaf None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:56.152792 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ae746-f2d2-4a88-9cb8-222c93aace51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:56.154428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ae746-f2d2-4a88-9cb8-222c93aace51 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:56.156245 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ae746-f2d2-4a88-9cb8-222c93aace51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:56.160343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9479d43-1cf4-4651-bf08-a1f28c9c8d77 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:56.160343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9479d43-1cf4-4651-bf08-a1f28c9c8d77 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:56.160343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9479d43-1cf4-4651-bf08-a1f28c9c8d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.162001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ae746-f2d2-4a88-9cb8-222c93aace51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.163694 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ae746-f2d2-4a88-9cb8-222c93aace51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:56.164850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ae746-f2d2-4a88-9cb8-222c93aace51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:56.169141 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ae746-f2d2-4a88-9cb8-222c93aace51 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:56.203565 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 772/1542] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:43:56.205001 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e9479d43-1cf4-4651-bf08-a1f28c9c8d77 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:56.206745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e9479d43-1cf4-4651-bf08-a1f28c9c8d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:56.222762 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ae746-f2d2-4a88-9cb8-222c93aace51 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:56.230987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 771/1543] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:56.240164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f358b56-670a-4a00-b9a5-610ddb69d49a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:56.240164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f358b56-670a-4a00-b9a5-610ddb69d49a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:56.240585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f358b56-670a-4a00-b9a5-610ddb69d49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.262778 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ae746-f2d2-4a88-9cb8-222c93aace51 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:56.279799 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede7abe-ce3f-4b86-8489-747ff6d4c0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:56.281497 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede7abe-ce3f-4b86-8489-747ff6d4c0c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:56.282977 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede7abe-ce3f-4b86-8489-747ff6d4c0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:56.283906 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede7abe-ce3f-4b86-8489-747ff6d4c0c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.285502 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede7abe-ce3f-4b86-8489-747ff6d4c0c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:56.286709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1f358b56-670a-4a00-b9a5-610ddb69d49a None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:56.288164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1f358b56-670a-4a00-b9a5-610ddb69d49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:56.289073 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: WARNING py.warnings [None req-4ede7abe-ce3f-4b86-8489-747ff6d4c0c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:43:56.289073 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: warnings.warn(msg) Aug 30 14:43:56.289073 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: Aug 30 14:43:56.296452 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede7abe-ce3f-4b86-8489-747ff6d4c0c7 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:56.309169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 772/1544] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:56.314530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95b75d-6834-482d-843a-829f203a4e59 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:56.314852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95b75d-6834-482d-843a-829f203a4e59 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:56.315205 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95b75d-6834-482d-843a-829f203a4e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.353908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2f95b75d-6834-482d-843a-829f203a4e59 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:56.355254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2f95b75d-6834-482d-843a-829f203a4e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:56.372270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 773/1545] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:56.373344 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95b75d-6834-482d-843a-829f203a4e59 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:56.382780 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 773/1546] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:56 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:56.387311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede7abe-ce3f-4b86-8489-747ff6d4c0c7 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:56.390823 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95b75d-6834-482d-843a-829f203a4e59 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:56.501908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79d4ac-8e9b-4450-aeab-e05671157748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:56.502461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79d4ac-8e9b-4450-aeab-e05671157748 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:56.502727 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79d4ac-8e9b-4450-aeab-e05671157748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:56.502904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede7abe-ce3f-4b86-8489-747ff6d4c0c7 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:56.503069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79d4ac-8e9b-4450-aeab-e05671157748 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.504772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79d4ac-8e9b-4450-aeab-e05671157748 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:56.504772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79d4ac-8e9b-4450-aeab-e05671157748 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:56.505172 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: WARNING py.warnings [None req-1d79d4ac-8e9b-4450-aeab-e05671157748 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:43:56.505172 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: warnings.warn(msg) Aug 30 14:43:56.505172 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: Aug 30 14:43:56.506089 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79d4ac-8e9b-4450-aeab-e05671157748 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:56.521064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30841e1-9a48-44a8-95da-080d9ee19777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:56.521729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30841e1-9a48-44a8-95da-080d9ee19777 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:56.522013 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30841e1-9a48-44a8-95da-080d9ee19777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:56.522286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30841e1-9a48-44a8-95da-080d9ee19777 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.522955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30841e1-9a48-44a8-95da-080d9ee19777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:56.525057 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30841e1-9a48-44a8-95da-080d9ee19777 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:56.526997 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 774/1547] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:56.532568 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c9a59-1290-444e-aaf6-e0fb090cdd97 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:56.532873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c9a59-1290-444e-aaf6-e0fb090cdd97 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:56.533211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c9a59-1290-444e-aaf6-e0fb090cdd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.575524 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 774/1548] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:56 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:56.576521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30841e1-9a48-44a8-95da-080d9ee19777 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:56.578819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-050c9a59-1290-444e-aaf6-e0fb090cdd97 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:56.600755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-050c9a59-1290-444e-aaf6-e0fb090cdd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:56.601460 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30841e1-9a48-44a8-95da-080d9ee19777 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:56.621880 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055afbb-d7fd-45b9-9dd2-597cbf330cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:56.622543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055afbb-d7fd-45b9-9dd2-597cbf330cda admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:56.622844 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055afbb-d7fd-45b9-9dd2-597cbf330cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:56.623171 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055afbb-d7fd-45b9-9dd2-597cbf330cda admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.623819 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055afbb-d7fd-45b9-9dd2-597cbf330cda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:56.626700 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055afbb-d7fd-45b9-9dd2-597cbf330cda admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:56.640147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 775/1549] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:56.643255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c9a59-1290-444e-aaf6-e0fb090cdd97 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:56.679763 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c9a59-1290-444e-aaf6-e0fb090cdd97 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:56.704312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 775/1550] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:56 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:56.705765 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055afbb-d7fd-45b9-9dd2-597cbf330cda admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:56.706592 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dd6b6-2b18-4e21-9a6c-d40a7ee25120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:56.708208 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dd6b6-2b18-4e21-9a6c-d40a7ee25120 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:56.709989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dd6b6-2b18-4e21-9a6c-d40a7ee25120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:56.711003 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dd6b6-2b18-4e21-9a6c-d40a7ee25120 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.712650 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813dd6b6-2b18-4e21-9a6c-d40a7ee25120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:56.716167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813dd6b6-2b18-4e21-9a6c-d40a7ee25120 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:56.767965 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 776/1551] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:56.770368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b35c0-c57f-41ba-b8fc-a4a268b010d9 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:56.770832 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b35c0-c57f-41ba-b8fc-a4a268b010d9 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:56.771505 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b35c0-c57f-41ba-b8fc-a4a268b010d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.830431 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-6f9b35c0-c57f-41ba-b8fc-a4a268b010d9 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:56.833849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6f9b35c0-c57f-41ba-b8fc-a4a268b010d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:56.920362 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055afbb-d7fd-45b9-9dd2-597cbf330cda admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:56.929428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fc580-f284-4c75-b627-6c22917e443f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:56.930812 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fc580-f284-4c75-b627-6c22917e443f admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:56.931608 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fc580-f284-4c75-b627-6c22917e443f admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:56.932177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fc580-f284-4c75-b627-6c22917e443f admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.933370 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fc580-f284-4c75-b627-6c22917e443f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:56.933905 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fc580-f284-4c75-b627-6c22917e443f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:56.937717 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fc580-f284-4c75-b627-6c22917e443f admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:56.966153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 776/1552] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:56.967789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fc580-f284-4c75-b627-6c22917e443f admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:56.974937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 777/1553] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:56.976767 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c067e-0fed-4ca8-a479-683f910e8acc None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:56.977076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c067e-0fed-4ca8-a479-683f910e8acc None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:56.977342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c067e-0fed-4ca8-a479-683f910e8acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.995842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fc580-f284-4c75-b627-6c22917e443f admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:57.010742 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa0e2a-a2b2-4354-9ac4-c9e4b027fb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:57.013046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa0e2a-a2b2-4354-9ac4-c9e4b027fb3a admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.013805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa0e2a-a2b2-4354-9ac4-c9e4b027fb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.014231 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa0e2a-a2b2-4354-9ac4-c9e4b027fb3a admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.015750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa0e2a-a2b2-4354-9ac4-c9e4b027fb3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:57.019318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f63c067e-0fed-4ca8-a479-683f910e8acc None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:57.021351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f63c067e-0fed-4ca8-a479-683f910e8acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:57.025743 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa0e2a-a2b2-4354-9ac4-c9e4b027fb3a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:57.036140 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 778/1554] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:57.038440 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c067e-0fed-4ca8-a479-683f910e8acc None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:57.046823 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:57.056165 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c067e-0fed-4ca8-a479-683f910e8acc None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:57.064511 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072cf4d-91c8-4409-9f80-97820145e6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:57.065685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072cf4d-91c8-4409-9f80-97820145e6ad admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.065685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072cf4d-91c8-4409-9f80-97820145e6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.066007 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072cf4d-91c8-4409-9f80-97820145e6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.066662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072cf4d-91c8-4409-9f80-97820145e6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:57.068840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072cf4d-91c8-4409-9f80-97820145e6ad admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:57.084029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 777/1555] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:56 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:57.084029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f42ad-9775-42e4-820a-07d2166a16ba None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.084029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f42ad-9775-42e4-820a-07d2166a16ba None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.084029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f42ad-9775-42e4-820a-07d2166a16ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.092039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 779/1556] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:57.093672 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc9e15-760d-4149-b74c-3e1eb1e6d416 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.094091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc9e15-760d-4149-b74c-3e1eb1e6d416 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.094391 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc9e15-760d-4149-b74c-3e1eb1e6d416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.106546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-740f42ad-9775-42e4-820a-07d2166a16ba None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:57.108086 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-740f42ad-9775-42e4-820a-07d2166a16ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:57.117094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 778/1557] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:57.118078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f42ad-9775-42e4-820a-07d2166a16ba None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:57.119441 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-befc9e15-760d-4149-b74c-3e1eb1e6d416 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:57.120747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-befc9e15-760d-4149-b74c-3e1eb1e6d416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:57.129797 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f42ad-9775-42e4-820a-07d2166a16ba None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:57.131440 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 780/1558] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:57.132721 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec5909-a64d-49c5-9dad-37e239976d32 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.133028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec5909-a64d-49c5-9dad-37e239976d32 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.133324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec5909-a64d-49c5-9dad-37e239976d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.135953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f425e57-0158-4e25-ad04-448d03578b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:57.137097 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f425e57-0158-4e25-ad04-448d03578b16 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.137374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f425e57-0158-4e25-ad04-448d03578b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.137642 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f425e57-0158-4e25-ad04-448d03578b16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.138332 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f425e57-0158-4e25-ad04-448d03578b16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:57.140442 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f425e57-0158-4e25-ad04-448d03578b16 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:57.156352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a6ec5909-a64d-49c5-9dad-37e239976d32 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:57.157699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a6ec5909-a64d-49c5-9dad-37e239976d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:57.167466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 781/1559] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:57.168595 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f36d8-e07f-423a-b2d8-11a640a6f498 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.168879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f36d8-e07f-423a-b2d8-11a640a6f498 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.169153 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f36d8-e07f-423a-b2d8-11a640a6f498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.178309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 779/1560] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:57 2022] POST /identity/v3/projects => generated 387 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:57.180326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980d0a5-f53d-4a84-866a-cbad159c108c None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.180326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980d0a5-f53d-4a84-866a-cbad159c108c None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.180670 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980d0a5-f53d-4a84-866a-cbad159c108c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.193528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1b0f36d8-e07f-423a-b2d8-11a640a6f498 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:57.195177 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1b0f36d8-e07f-423a-b2d8-11a640a6f498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:57.204862 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-f980d0a5-f53d-4a84-866a-cbad159c108c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:57.206147 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f980d0a5-f53d-4a84-866a-cbad159c108c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:57.212275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:57.272389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 782/1561] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:57.273709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f36d8-e07f-423a-b2d8-11a640a6f498 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:57.284255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f36d8-e07f-423a-b2d8-11a640a6f498 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:57.289558 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2542ccf-5982-47df-8bce-29044707847a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:57.290291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2542ccf-5982-47df-8bce-29044707847a admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.290571 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2542ccf-5982-47df-8bce-29044707847a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.290837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2542ccf-5982-47df-8bce-29044707847a admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.291500 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2542ccf-5982-47df-8bce-29044707847a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:57.292515 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 780/1562] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:57.293198 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2542ccf-5982-47df-8bce-29044707847a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:57.294039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f980d0a5-f53d-4a84-866a-cbad159c108c None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:57.305735 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 783/1563] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:57.307020 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a7d10-6c6a-42f3-b0af-c3c4eb62f0ea None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.307306 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a7d10-6c6a-42f3-b0af-c3c4eb62f0ea None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.307604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a7d10-6c6a-42f3-b0af-c3c4eb62f0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.314941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f980d0a5-f53d-4a84-866a-cbad159c108c None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:57.321494 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fcb9b-9fa5-43e3-8a70-6dfeed0aa1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:57.322300 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fcb9b-9fa5-43e3-8a70-6dfeed0aa1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.322577 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fcb9b-9fa5-43e3-8a70-6dfeed0aa1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.322840 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fcb9b-9fa5-43e3-8a70-6dfeed0aa1b3 admin admin] PATH_INFO: `/v3/projects/a49024fae001434a996e45222afbb603/users/1df8719693204e4f8737478cba11dc31/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.334745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e87a7d10-6c6a-42f3-b0af-c3c4eb62f0ea None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:57.336459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e87a7d10-6c6a-42f3-b0af-c3c4eb62f0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:57.343049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fcb9b-9fa5-43e3-8a70-6dfeed0aa1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49024fae001434a996e45222afbb603, user_id=1df8719693204e4f8737478cba11dc31, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:57.344140 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: WARNING py.warnings [None req-319fcb9b-9fa5-43e3-8a70-6dfeed0aa1b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:43:57.344140 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: warnings.warn(msg) Aug 30 14:43:57.344140 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: Aug 30 14:43:57.346204 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fcb9b-9fa5-43e3-8a70-6dfeed0aa1b3 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:57.347404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 784/1564] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:57.349290 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a7d10-6c6a-42f3-b0af-c3c4eb62f0ea None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:57.367041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 781/1565] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:57 2022] PUT /identity/v3/projects/a49024fae001434a996e45222afbb603/users/1df8719693204e4f8737478cba11dc31/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:57.367360 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a7d10-6c6a-42f3-b0af-c3c4eb62f0ea None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:57.368832 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee4a4d-5544-4f07-b584-ffc985ef8aec None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.369064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee4a4d-5544-4f07-b584-ffc985ef8aec None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.369292 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee4a4d-5544-4f07-b584-ffc985ef8aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.396456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-5cee4a4d-5544-4f07-b584-ffc985ef8aec None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:57.398264 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5cee4a4d-5544-4f07-b584-ffc985ef8aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:57.449183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2e799-4c1c-42cf-9311-3610cd366a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:57.450340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2e799-4c1c-42cf-9311-3610cd366a3e admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.450788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2e799-4c1c-42cf-9311-3610cd366a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.451231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2e799-4c1c-42cf-9311-3610cd366a3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.452137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2e799-4c1c-42cf-9311-3610cd366a3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:57.452545 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2e799-4c1c-42cf-9311-3610cd366a3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:57.454624 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2e799-4c1c-42cf-9311-3610cd366a3e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:57.469080 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 785/1566] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:57.470479 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2e799-4c1c-42cf-9311-3610cd366a3e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:57.478342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 782/1567] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:57.479707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee4a4d-5544-4f07-b584-ffc985ef8aec None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:57.484189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2e799-4c1c-42cf-9311-3610cd366a3e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:57.489232 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccedd712-5f4d-4f81-98e4-4f6deb66d340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:57.489878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedd712-5f4d-4f81-98e4-4f6deb66d340 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.490155 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedd712-5f4d-4f81-98e4-4f6deb66d340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.490599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedd712-5f4d-4f81-98e4-4f6deb66d340 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.490912 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee4a4d-5544-4f07-b584-ffc985ef8aec None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:57.491258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccedd712-5f4d-4f81-98e4-4f6deb66d340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:57.492360 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: WARNING py.warnings [None req-ccedd712-5f4d-4f81-98e4-4f6deb66d340 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:43:57.492360 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: warnings.warn(msg) Aug 30 14:43:57.492360 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: Aug 30 14:43:57.493610 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccedd712-5f4d-4f81-98e4-4f6deb66d340 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:57.495938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32292c7e-4ebc-49f8-8f02-7ff1ff96fc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:57.496621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32292c7e-4ebc-49f8-8f02-7ff1ff96fc85 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.496903 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32292c7e-4ebc-49f8-8f02-7ff1ff96fc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.497173 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32292c7e-4ebc-49f8-8f02-7ff1ff96fc85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.497856 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32292c7e-4ebc-49f8-8f02-7ff1ff96fc85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:57.498154 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32292c7e-4ebc-49f8-8f02-7ff1ff96fc85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:57.500155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32292c7e-4ebc-49f8-8f02-7ff1ff96fc85 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:57.513900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 783/1568] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:57.520602 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 786/1569] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:57 2022] POST /identity/v3/users => generated 294 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:57.521907 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccedd712-5f4d-4f81-98e4-4f6deb66d340 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:57.523051 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3511a9-7f4a-4443-8862-08304eef266d None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.524095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3511a9-7f4a-4443-8862-08304eef266d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.526229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3511a9-7f4a-4443-8862-08304eef266d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.534891 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccedd712-5f4d-4f81-98e4-4f6deb66d340 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:57.541691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8441b0b-f258-472e-a3ad-a5b8230a3e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:57.542837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8441b0b-f258-472e-a3ad-a5b8230a3e65 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.543111 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8441b0b-f258-472e-a3ad-a5b8230a3e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.543408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8441b0b-f258-472e-a3ad-a5b8230a3e65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.544149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8441b0b-f258-472e-a3ad-a5b8230a3e65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:57.546282 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8441b0b-f258-472e-a3ad-a5b8230a3e65 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:57.553617 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-2e3511a9-7f4a-4443-8862-08304eef266d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:57.555685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2e3511a9-7f4a-4443-8862-08304eef266d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:57.560323 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 787/1570] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:57.560859 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8441b0b-f258-472e-a3ad-a5b8230a3e65 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:57.570665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 784/1571] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:57.572548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45361a-ce4e-4d62-8022-dcbe99fc413d None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.572905 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45361a-ce4e-4d62-8022-dcbe99fc413d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.573287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45361a-ce4e-4d62-8022-dcbe99fc413d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.576990 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8441b0b-f258-472e-a3ad-a5b8230a3e65 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:57.587466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f1e38-8d28-4689-bad9-b526a69db88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:57.587958 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f1e38-8d28-4689-bad9-b526a69db88a admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.588282 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f1e38-8d28-4689-bad9-b526a69db88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.589467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f1e38-8d28-4689-bad9-b526a69db88a admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.589467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f1e38-8d28-4689-bad9-b526a69db88a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:57.589778 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f1e38-8d28-4689-bad9-b526a69db88a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:57.591515 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f1e38-8d28-4689-bad9-b526a69db88a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:57.609004 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 788/1572] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:57.610481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f1e38-8d28-4689-bad9-b526a69db88a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:57.622217 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-7c45361a-ce4e-4d62-8022-dcbe99fc413d None None] MFA Rules not processed for user `1df8719693204e4f8737478cba11dc31`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:57.624537 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7c45361a-ce4e-4d62-8022-dcbe99fc413d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:57.625455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f1e38-8d28-4689-bad9-b526a69db88a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:57.633519 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca541d-6c82-4e2d-b4c8-42ec50c8e752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:57.634694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca541d-6c82-4e2d-b4c8-42ec50c8e752 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.635137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca541d-6c82-4e2d-b4c8-42ec50c8e752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.635647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca541d-6c82-4e2d-b4c8-42ec50c8e752 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.636725 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca541d-6c82-4e2d-b4c8-42ec50c8e752 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:57.640306 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca541d-6c82-4e2d-b4c8-42ec50c8e752 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:57.642226 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:57.682762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 789/1573] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:57 2022] POST /identity/v3/projects => generated 379 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:57.683634 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca541d-6c82-4e2d-b4c8-42ec50c8e752 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:57.725884 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 785/1574] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:57.726237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:57.726402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45361a-ce4e-4d62-8022-dcbe99fc413d None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:57.740901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45361a-ce4e-4d62-8022-dcbe99fc413d None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:57.761908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca541d-6c82-4e2d-b4c8-42ec50c8e752 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:57.767364 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8e961-df48-40e0-bc18-7179e718e5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:57.768029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8e961-df48-40e0-bc18-7179e718e5ea admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.768325 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8e961-df48-40e0-bc18-7179e718e5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.768740 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8e961-df48-40e0-bc18-7179e718e5ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.769415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b8e961-df48-40e0-bc18-7179e718e5ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:57.769715 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b8e961-df48-40e0-bc18-7179e718e5ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:57.771404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b8e961-df48-40e0-bc18-7179e718e5ea admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:57.785512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 790/1575] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:57.786399 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8e961-df48-40e0-bc18-7179e718e5ea admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:57.797932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8e961-df48-40e0-bc18-7179e718e5ea admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:57.803086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a21639-600d-4dec-8b75-46baabd1cdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:57.803908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a21639-600d-4dec-8b75-46baabd1cdd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.804285 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a21639-600d-4dec-8b75-46baabd1cdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.804629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a21639-600d-4dec-8b75-46baabd1cdd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.805378 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a21639-600d-4dec-8b75-46baabd1cdd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:57.807055 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a21639-600d-4dec-8b75-46baabd1cdd9 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:57.812437 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5597d1-06e2-4994-8135-0537ec8a0830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:57.813638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5597d1-06e2-4994-8135-0537ec8a0830 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.813924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5597d1-06e2-4994-8135-0537ec8a0830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.814196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5597d1-06e2-4994-8135-0537ec8a0830 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.814857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5597d1-06e2-4994-8135-0537ec8a0830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:57.816875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5597d1-06e2-4994-8135-0537ec8a0830 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:57.819038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 791/1576] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:57.822943 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a21639-600d-4dec-8b75-46baabd1cdd9 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:57.834168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a21639-600d-4dec-8b75-46baabd1cdd9 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:57.840638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2b42f-4204-49bb-8f19-502ba82b2ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:57.841335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2b42f-4204-49bb-8f19-502ba82b2ab3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.841611 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2b42f-4204-49bb-8f19-502ba82b2ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.841882 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2b42f-4204-49bb-8f19-502ba82b2ab3 admin admin] PATH_INFO: `/v3/projects/bc68c4d868d645508a5ab248bc58481a/users/c875c46a88e04b839dee75daf10c156d/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.851770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 786/1577] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:57 2022] POST /identity/v3/projects => generated 393 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:57.852959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5597d1-06e2-4994-8135-0537ec8a0830 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:57.860603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2b42f-4204-49bb-8f19-502ba82b2ab3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc68c4d868d645508a5ab248bc58481a, user_id=c875c46a88e04b839dee75daf10c156d, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:57.863462 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2b42f-4204-49bb-8f19-502ba82b2ab3 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:57.880193 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 792/1578] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:57 2022] PUT /identity/v3/projects/bc68c4d868d645508a5ab248bc58481a/users/c875c46a88e04b839dee75daf10c156d/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:57.888947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2b42f-4204-49bb-8f19-502ba82b2ab3 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:57.932238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5597d1-06e2-4994-8135-0537ec8a0830 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:57.968321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2b42f-4204-49bb-8f19-502ba82b2ab3 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:57.974235 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febed460-eeb7-4afe-a8dd-c1b4065995e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:57.974934 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febed460-eeb7-4afe-a8dd-c1b4065995e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:57.975262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febed460-eeb7-4afe-a8dd-c1b4065995e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:57.975597 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febed460-eeb7-4afe-a8dd-c1b4065995e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:57.976370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febed460-eeb7-4afe-a8dd-c1b4065995e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:57.978716 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febed460-eeb7-4afe-a8dd-c1b4065995e1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:57.983381 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:58.003711 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06663f56-7abe-4a65-aab7-5fd55a36d92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:58.004719 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06663f56-7abe-4a65-aab7-5fd55a36d92d admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.004719 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06663f56-7abe-4a65-aab7-5fd55a36d92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.005050 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06663f56-7abe-4a65-aab7-5fd55a36d92d admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.005773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06663f56-7abe-4a65-aab7-5fd55a36d92d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:58.006077 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06663f56-7abe-4a65-aab7-5fd55a36d92d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:58.007744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06663f56-7abe-4a65-aab7-5fd55a36d92d admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:58.013403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 793/1579] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:57 2022] POST /identity/v3/projects => generated 365 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:58.018312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febed460-eeb7-4afe-a8dd-c1b4065995e1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:58.021584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 787/1580] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:58.022464 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06663f56-7abe-4a65-aab7-5fd55a36d92d admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:58.033189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06663f56-7abe-4a65-aab7-5fd55a36d92d admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:58.103226 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febed460-eeb7-4afe-a8dd-c1b4065995e1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:58.103789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5365f96-11e5-45d9-9d16-4542e0df3cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:58.104481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5365f96-11e5-45d9-9d16-4542e0df3cec admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.105037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5365f96-11e5-45d9-9d16-4542e0df3cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.105385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5365f96-11e5-45d9-9d16-4542e0df3cec admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.106358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5365f96-11e5-45d9-9d16-4542e0df3cec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:58.108520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0be07-692a-464e-b1e7-c4b3c2cc43d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:58.108850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5365f96-11e5-45d9-9d16-4542e0df3cec admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:58.109427 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0be07-692a-464e-b1e7-c4b3c2cc43d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.109701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0be07-692a-464e-b1e7-c4b3c2cc43d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.110162 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0be07-692a-464e-b1e7-c4b3c2cc43d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.110828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0be07-692a-464e-b1e7-c4b3c2cc43d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:58.112867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0be07-692a-464e-b1e7-c4b3c2cc43d2 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:58.143647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 794/1581] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:58 2022] POST /identity/v3/users => generated 290 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:58.144572 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0be07-692a-464e-b1e7-c4b3c2cc43d2 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:58.150919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 788/1582] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:58 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:58.150919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5365f96-11e5-45d9-9d16-4542e0df3cec admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:58.155286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0be07-692a-464e-b1e7-c4b3c2cc43d2 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:58.160075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:58.228555 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5365f96-11e5-45d9-9d16-4542e0df3cec admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:58.236362 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e72a6f-f91e-4885-85e9-431fc1c7e269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:58.237421 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e72a6f-f91e-4885-85e9-431fc1c7e269 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.237858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e72a6f-f91e-4885-85e9-431fc1c7e269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.238269 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e72a6f-f91e-4885-85e9-431fc1c7e269 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.238402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e0fd3-2b3e-4c53-9d07-74434f96bbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:58.239301 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e0fd3-2b3e-4c53-9d07-74434f96bbe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.239442 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e72a6f-f91e-4885-85e9-431fc1c7e269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:58.239772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e0fd3-2b3e-4c53-9d07-74434f96bbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.240252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e0fd3-2b3e-4c53-9d07-74434f96bbe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.241319 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e0fd3-2b3e-4c53-9d07-74434f96bbe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:58.242931 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e72a6f-f91e-4885-85e9-431fc1c7e269 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:58.244201 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e0fd3-2b3e-4c53-9d07-74434f96bbe8 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:58.263054 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 789/1583] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:58.264332 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e0fd3-2b3e-4c53-9d07-74434f96bbe8 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:58.278787 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e0fd3-2b3e-4c53-9d07-74434f96bbe8 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:58.285478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467258b4-fd46-48f7-953e-2806561a7908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:58.286460 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467258b4-fd46-48f7-953e-2806561a7908 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.286893 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467258b4-fd46-48f7-953e-2806561a7908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.287331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467258b4-fd46-48f7-953e-2806561a7908 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.288245 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467258b4-fd46-48f7-953e-2806561a7908 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:58.290146 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 795/1584] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:58 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:58.290332 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467258b4-fd46-48f7-953e-2806561a7908 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:58.291651 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f7371-fdbe-466f-ad11-72d0c0d1705f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.291955 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f7371-fdbe-466f-ad11-72d0c0d1705f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.292225 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f7371-fdbe-466f-ad11-72d0c0d1705f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.315674 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 790/1585] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:58 2022] POST /identity/v3/users => generated 297 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:58.316595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467258b4-fd46-48f7-953e-2806561a7908 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:58.327915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467258b4-fd46-48f7-953e-2806561a7908 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:58.330438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-cd0f7371-fdbe-466f-ad11-72d0c0d1705f None None] MFA Rules not processed for user `c875c46a88e04b839dee75daf10c156d`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:58.331803 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cd0f7371-fdbe-466f-ad11-72d0c0d1705f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:58.401057 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:58.443768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 796/1586] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:58.444115 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ce7f0-12d0-48f7-9809-3610b40f94c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:58.444115 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ce7f0-12d0-48f7-9809-3610b40f94c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.444581 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f7371-fdbe-466f-ad11-72d0c0d1705f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:58.444740 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ce7f0-12d0-48f7-9809-3610b40f94c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.444740 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ce7f0-12d0-48f7-9809-3610b40f94c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.444740 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ce7f0-12d0-48f7-9809-3610b40f94c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:58.444740 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ce7f0-12d0-48f7-9809-3610b40f94c3 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:58.458889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f7371-fdbe-466f-ad11-72d0c0d1705f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:58.466569 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a710f81-7d83-4b63-8749-5dbf193b7225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:58.467657 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a710f81-7d83-4b63-8749-5dbf193b7225 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.468409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a710f81-7d83-4b63-8749-5dbf193b7225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.469684 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a710f81-7d83-4b63-8749-5dbf193b7225 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.470782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a710f81-7d83-4b63-8749-5dbf193b7225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:58.474233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a710f81-7d83-4b63-8749-5dbf193b7225 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:58.480235 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 791/1587] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:58 2022] POST /identity/v3/users => generated 283 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:58.480235 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ce7f0-12d0-48f7-9809-3610b40f94c3 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:58.496182 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ce7f0-12d0-48f7-9809-3610b40f94c3 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:58.504483 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a0aec-ab03-43e4-a19f-3aaf8fdb996f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:58.505336 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a0aec-ab03-43e4-a19f-3aaf8fdb996f admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.505691 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a0aec-ab03-43e4-a19f-3aaf8fdb996f admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.506075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a0aec-ab03-43e4-a19f-3aaf8fdb996f admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.507104 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a0aec-ab03-43e4-a19f-3aaf8fdb996f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:58.510548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a0aec-ab03-43e4-a19f-3aaf8fdb996f admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:58.520585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 797/1588] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:58 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:58.522213 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a710f81-7d83-4b63-8749-5dbf193b7225 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:58.526214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 792/1589] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:58.527444 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a0aec-ab03-43e4-a19f-3aaf8fdb996f admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:58.541579 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a0aec-ab03-43e4-a19f-3aaf8fdb996f admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:58.614155 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a710f81-7d83-4b63-8749-5dbf193b7225 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:58.619221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb88c9-2eaf-414a-b304-f4b77c52bb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:58.619858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb88c9-2eaf-414a-b304-f4b77c52bb60 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.620173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb88c9-2eaf-414a-b304-f4b77c52bb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.620448 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb88c9-2eaf-414a-b304-f4b77c52bb60 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.621102 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb88c9-2eaf-414a-b304-f4b77c52bb60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:58.622725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c00b6-eec9-4160-8d9d-b69dad79c25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:58.623533 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c00b6-eec9-4160-8d9d-b69dad79c25b admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.623782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb88c9-2eaf-414a-b304-f4b77c52bb60 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:58.624255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c00b6-eec9-4160-8d9d-b69dad79c25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.625439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c00b6-eec9-4160-8d9d-b69dad79c25b admin admin] PATH_INFO: `/v3/projects/21f6e5be24e448f9b981ad77d23de8bd/users/521cc5ea8fb045bc81e8e3bfa8ba239b/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.633501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:58.650027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c00b6-eec9-4160-8d9d-b69dad79c25b admin admin] RBAC: Authorizing `identity:create_grant(project_id=21f6e5be24e448f9b981ad77d23de8bd, user_id=521cc5ea8fb045bc81e8e3bfa8ba239b, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:58.653917 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c00b6-eec9-4160-8d9d-b69dad79c25b admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:58.655261 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 798/1590] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:58 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:58.656221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb88c9-2eaf-414a-b304-f4b77c52bb60 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:58.674877 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 793/1591] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:58 2022] PUT /identity/v3/projects/21f6e5be24e448f9b981ad77d23de8bd/users/521cc5ea8fb045bc81e8e3bfa8ba239b/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:58.675570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb88c9-2eaf-414a-b304-f4b77c52bb60 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:58.676311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c00b6-eec9-4160-8d9d-b69dad79c25b admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:58.762189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c00b6-eec9-4160-8d9d-b69dad79c25b admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:58.767974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a5234-12e1-4a72-b0fd-521c068110e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:58.768646 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a5234-12e1-4a72-b0fd-521c068110e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.768924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a5234-12e1-4a72-b0fd-521c068110e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.769191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a5234-12e1-4a72-b0fd-521c068110e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.769864 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a5234-12e1-4a72-b0fd-521c068110e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:58.771566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a5234-12e1-4a72-b0fd-521c068110e9 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:58.784789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c54c23-7e81-4f4d-ab48-e4434ba6af9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:58.785412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c54c23-7e81-4f4d-ab48-e4434ba6af9e admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.785694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c54c23-7e81-4f4d-ab48-e4434ba6af9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.785953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c54c23-7e81-4f4d-ab48-e4434ba6af9e admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.786605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c54c23-7e81-4f4d-ab48-e4434ba6af9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:58.788637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c54c23-7e81-4f4d-ab48-e4434ba6af9e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:58.790964 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 794/1592] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:58.792379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a5234-12e1-4a72-b0fd-521c068110e9 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:58.808679 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a5234-12e1-4a72-b0fd-521c068110e9 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:58.816483 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e719b82-92dd-4dcf-9b64-fb21fd2dd0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:58.817295 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e719b82-92dd-4dcf-9b64-fb21fd2dd0e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.817734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e719b82-92dd-4dcf-9b64-fb21fd2dd0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.818111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e719b82-92dd-4dcf-9b64-fb21fd2dd0e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.818881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e719b82-92dd-4dcf-9b64-fb21fd2dd0e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:58.819034 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 799/1593] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:58 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:58.820583 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c54c23-7e81-4f4d-ab48-e4434ba6af9e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:58.822584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e719b82-92dd-4dcf-9b64-fb21fd2dd0e3 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:58.835618 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c54c23-7e81-4f4d-ab48-e4434ba6af9e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:58.842223 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f9401-b42b-4202-8bd1-26ede69f0aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:58.843546 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f9401-b42b-4202-8bd1-26ede69f0aef admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.844155 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f9401-b42b-4202-8bd1-26ede69f0aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.844587 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f9401-b42b-4202-8bd1-26ede69f0aef admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.845772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f9401-b42b-4202-8bd1-26ede69f0aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:58.848899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f9401-b42b-4202-8bd1-26ede69f0aef admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:58.865025 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 795/1594] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:58 2022] POST /identity/v3/projects => generated 381 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:58.866411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e719b82-92dd-4dcf-9b64-fb21fd2dd0e3 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:58.871960 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 800/1595] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:58.871960 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f9401-b42b-4202-8bd1-26ede69f0aef admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:58.886974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f9401-b42b-4202-8bd1-26ede69f0aef admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:58.893289 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:58.956223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e719b82-92dd-4dcf-9b64-fb21fd2dd0e3 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:58.962266 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87ae79-bc3a-4e71-a495-f5cdf32b12d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:58.963149 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87ae79-bc3a-4e71-a495-f5cdf32b12d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.963525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87ae79-bc3a-4e71-a495-f5cdf32b12d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.963904 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87ae79-bc3a-4e71-a495-f5cdf32b12d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.964726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d87ae79-bc3a-4e71-a495-f5cdf32b12d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:58.966888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d87ae79-bc3a-4e71-a495-f5cdf32b12d8 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:58.977359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488606b-0d15-42df-acaf-887db01e4083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:58.978742 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488606b-0d15-42df-acaf-887db01e4083 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:58.979165 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488606b-0d15-42df-acaf-887db01e4083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:58.980032 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488606b-0d15-42df-acaf-887db01e4083 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:58.981483 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488606b-0d15-42df-acaf-887db01e4083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:58.984877 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488606b-0d15-42df-acaf-887db01e4083 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:59.012635 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 801/1596] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:59.012635 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488606b-0d15-42df-acaf-887db01e4083 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:59.013115 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 796/1597] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:58 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:59.013115 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d684dd4-8681-451b-92c1-c41061ce446a None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.013115 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d684dd4-8681-451b-92c1-c41061ce446a None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.013115 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d684dd4-8681-451b-92c1-c41061ce446a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.019630 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488606b-0d15-42df-acaf-887db01e4083 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:59.025655 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683780c-0e94-47a2-af08-04584239afa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:59.026393 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683780c-0e94-47a2-af08-04584239afa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.026784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683780c-0e94-47a2-af08-04584239afa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.027210 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683780c-0e94-47a2-af08-04584239afa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.028182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683780c-0e94-47a2-af08-04584239afa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:59.030880 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683780c-0e94-47a2-af08-04584239afa8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:59.043678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-7d684dd4-8681-451b-92c1-c41061ce446a None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:59.045385 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 802/1598] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:59.046044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7d684dd4-8681-451b-92c1-c41061ce446a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:59.046665 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683780c-0e94-47a2-af08-04584239afa8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:59.060387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683780c-0e94-47a2-af08-04584239afa8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:59.061184 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 797/1599] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:59.062949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d684dd4-8681-451b-92c1-c41061ce446a None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:59.069292 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e27314-60c0-4de7-b3a7-f3117f349eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:59.070256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e27314-60c0-4de7-b3a7-f3117f349eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.070636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e27314-60c0-4de7-b3a7-f3117f349eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.071008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e27314-60c0-4de7-b3a7-f3117f349eb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.071778 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e27314-60c0-4de7-b3a7-f3117f349eb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:59.073686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e27314-60c0-4de7-b3a7-f3117f349eb3 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:59.080667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d684dd4-8681-451b-92c1-c41061ce446a None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:59.085611 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 803/1600] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:59.086508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e27314-60c0-4de7-b3a7-f3117f349eb3 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:59.089465 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd383fa-4150-4644-b467-e6a105563625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:59.090467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd383fa-4150-4644-b467-e6a105563625 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.090890 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd383fa-4150-4644-b467-e6a105563625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.091307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd383fa-4150-4644-b467-e6a105563625 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.092412 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd383fa-4150-4644-b467-e6a105563625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:59.095342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd383fa-4150-4644-b467-e6a105563625 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:59.098423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e27314-60c0-4de7-b3a7-f3117f349eb3 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:59.104334 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bfa3b-26eb-4d5c-a1ce-5cc75e368eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:59.105054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bfa3b-26eb-4d5c-a1ce-5cc75e368eaf admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.105258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bfa3b-26eb-4d5c-a1ce-5cc75e368eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.105526 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bfa3b-26eb-4d5c-a1ce-5cc75e368eaf admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.106193 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bfa3b-26eb-4d5c-a1ce-5cc75e368eaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:59.108264 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bfa3b-26eb-4d5c-a1ce-5cc75e368eaf admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:59.111965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 798/1601] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:59.112976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd383fa-4150-4644-b467-e6a105563625 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:59.126766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd383fa-4150-4644-b467-e6a105563625 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:59.132258 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd313a3-a666-4b91-ac3b-308268649657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:59.132890 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd313a3-a666-4b91-ac3b-308268649657 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.133164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd313a3-a666-4b91-ac3b-308268649657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.133434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd313a3-a666-4b91-ac3b-308268649657 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.134112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd313a3-a666-4b91-ac3b-308268649657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:59.135783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd313a3-a666-4b91-ac3b-308268649657 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:59.138763 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 804/1602] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:59 2022] POST /identity/v3/users => generated 297 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:59.140404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bfa3b-26eb-4d5c-a1ce-5cc75e368eaf admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:59.157549 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 799/1603] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:59.158186 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bfa3b-26eb-4d5c-a1ce-5cc75e368eaf admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:59.159597 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd313a3-a666-4b91-ac3b-308268649657 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:59.166427 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e6179-8eaa-47e3-8861-14cad94cf6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:59.167594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e6179-8eaa-47e3-8861-14cad94cf6af admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.168147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e6179-8eaa-47e3-8861-14cad94cf6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.168637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e6179-8eaa-47e3-8861-14cad94cf6af admin admin] PATH_INFO: `/v3/projects/7914879d13ad4a589f93b0ce22538d06/users/ee02494e189d4823ab776ea3da9fd2ee/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.176695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd313a3-a666-4b91-ac3b-308268649657 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:59.184561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7059e-0f7e-4631-9244-0ce664defaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:59.185506 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7059e-0f7e-4631-9244-0ce664defaa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.185919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7059e-0f7e-4631-9244-0ce664defaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.186307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7059e-0f7e-4631-9244-0ce664defaa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.187505 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7059e-0f7e-4631-9244-0ce664defaa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:59.190460 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7059e-0f7e-4631-9244-0ce664defaa4 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:59.191208 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e6179-8eaa-47e3-8861-14cad94cf6af admin admin] RBAC: Authorizing `identity:create_grant(project_id=7914879d13ad4a589f93b0ce22538d06, user_id=ee02494e189d4823ab776ea3da9fd2ee, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:59.194118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e6179-8eaa-47e3-8861-14cad94cf6af admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:59.205918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:INSERT=10,keystone:SELECT=90 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:59.210692 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 800/1604] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:59.212076 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7059e-0f7e-4631-9244-0ce664defaa4 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:59.212946 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 805/1605] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:59 2022] PUT /identity/v3/projects/7914879d13ad4a589f93b0ce22538d06/users/ee02494e189d4823ab776ea3da9fd2ee/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:59.213912 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e6179-8eaa-47e3-8861-14cad94cf6af admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:59.227820 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7059e-0f7e-4631-9244-0ce664defaa4 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:59.292729 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e6179-8eaa-47e3-8861-14cad94cf6af admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:59.299864 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cdf74c-1c36-4f7f-8d59-6b33f0989638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:59.300682 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdf74c-1c36-4f7f-8d59-6b33f0989638 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.301430 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdf74c-1c36-4f7f-8d59-6b33f0989638 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.301856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdf74c-1c36-4f7f-8d59-6b33f0989638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.314398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2eb36-ee84-4d09-b41f-ee81fc048c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:59.315476 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2eb36-ee84-4d09-b41f-ee81fc048c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.315919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2eb36-ee84-4d09-b41f-ee81fc048c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.316385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2eb36-ee84-4d09-b41f-ee81fc048c57 admin admin] PATH_INFO: `/v3/projects/144a5ee24aa44d4392da313cd405c715/users/2ba9fc062a1745eabc289b5586517b48/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.344791 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2eb36-ee84-4d09-b41f-ee81fc048c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=144a5ee24aa44d4392da313cd405c715, user_id=2ba9fc062a1745eabc289b5586517b48, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:59.348922 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2eb36-ee84-4d09-b41f-ee81fc048c57 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:59.369749 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 801/1606] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:59 2022] PUT /identity/v3/projects/144a5ee24aa44d4392da313cd405c715/users/2ba9fc062a1745eabc289b5586517b48/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:59.370768 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2eb36-ee84-4d09-b41f-ee81fc048c57 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:59.398532 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cdf74c-1c36-4f7f-8d59-6b33f0989638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:59.400142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cdf74c-1c36-4f7f-8d59-6b33f0989638 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:59.412731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 806/1607] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:59.413614 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cdf74c-1c36-4f7f-8d59-6b33f0989638 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:59.427213 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cdf74c-1c36-4f7f-8d59-6b33f0989638 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:59.459259 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:59.475280 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2eb36-ee84-4d09-b41f-ee81fc048c57 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:59.481945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf43e66-c687-4e25-a1a8-26c5521910e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:59.482875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf43e66-c687-4e25-a1a8-26c5521910e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.483161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf43e66-c687-4e25-a1a8-26c5521910e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.483640 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf43e66-c687-4e25-a1a8-26c5521910e6 admin admin] PATH_INFO: `/v3/projects/52028232fa7e4d808057e3e5512e6135/users/3398704b0dbc4af08e94b774a88851a4/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.505842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf43e66-c687-4e25-a1a8-26c5521910e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52028232fa7e4d808057e3e5512e6135, user_id=3398704b0dbc4af08e94b774a88851a4, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:59.509396 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf43e66-c687-4e25-a1a8-26c5521910e6 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:59.527338 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7421f2a-1928-4548-ac13-5d23b2c85b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:59.528363 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7421f2a-1928-4548-ac13-5d23b2c85b5e admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.528915 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7421f2a-1928-4548-ac13-5d23b2c85b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.529540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7421f2a-1928-4548-ac13-5d23b2c85b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.530448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 802/1608] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:59 2022] PUT /identity/v3/projects/52028232fa7e4d808057e3e5512e6135/users/3398704b0dbc4af08e94b774a88851a4/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:59.536759 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf43e66-c687-4e25-a1a8-26c5521910e6 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:59.537099 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7421f2a-1928-4548-ac13-5d23b2c85b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:59.537099 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7421f2a-1928-4548-ac13-5d23b2c85b5e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:59.555168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 807/1609] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:59.556543 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7421f2a-1928-4548-ac13-5d23b2c85b5e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:59.568836 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7421f2a-1928-4548-ac13-5d23b2c85b5e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:59.654941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf43e66-c687-4e25-a1a8-26c5521910e6 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:59.661763 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:59.665401 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34897292-8978-4b54-a3fe-e60225afa2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:59.666737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34897292-8978-4b54-a3fe-e60225afa2f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.667113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34897292-8978-4b54-a3fe-e60225afa2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.667455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34897292-8978-4b54-a3fe-e60225afa2f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.668323 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34897292-8978-4b54-a3fe-e60225afa2f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:59.670339 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34897292-8978-4b54-a3fe-e60225afa2f4 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:59.690966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 803/1610] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:59.692184 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34897292-8978-4b54-a3fe-e60225afa2f4 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:59.702467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43ed54-1543-44dc-a044-3f44d3aa9862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:59.702467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43ed54-1543-44dc-a044-3f44d3aa9862 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.702467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43ed54-1543-44dc-a044-3f44d3aa9862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.702467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43ed54-1543-44dc-a044-3f44d3aa9862 admin admin] PATH_INFO: `/v3/projects/561fa2cd4eb340358851628d57a98118/users/737e843d22c84e668b03e1437921cf05/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.709503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34897292-8978-4b54-a3fe-e60225afa2f4 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:59.715742 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7ccae-a6ba-44b1-8f7a-6c07deb3e40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:59.716438 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7ccae-a6ba-44b1-8f7a-6c07deb3e40a admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.717537 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7ccae-a6ba-44b1-8f7a-6c07deb3e40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.718061 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7ccae-a6ba-44b1-8f7a-6c07deb3e40a admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.719136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7ccae-a6ba-44b1-8f7a-6c07deb3e40a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:59.721400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7ccae-a6ba-44b1-8f7a-6c07deb3e40a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:59.732502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43ed54-1543-44dc-a044-3f44d3aa9862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=561fa2cd4eb340358851628d57a98118, user_id=737e843d22c84e668b03e1437921cf05, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:59.737976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43ed54-1543-44dc-a044-3f44d3aa9862 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:59.738821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 804/1611] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:59.740177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974d41d-e1ab-441e-88ee-89ce98a989b7 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.740538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974d41d-e1ab-441e-88ee-89ce98a989b7 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.740939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974d41d-e1ab-441e-88ee-89ce98a989b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.765093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 808/1612] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:59 2022] PUT /identity/v3/projects/561fa2cd4eb340358851628d57a98118/users/737e843d22c84e668b03e1437921cf05/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:59.767424 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43ed54-1543-44dc-a044-3f44d3aa9862 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:59.786227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-3974d41d-e1ab-441e-88ee-89ce98a989b7 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:59.788891 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-3974d41d-e1ab-441e-88ee-89ce98a989b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:59.887180 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43ed54-1543-44dc-a044-3f44d3aa9862 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:59.897415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3978c6-dbdb-479a-9bc3-7cfb5a061950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:59.898805 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3978c6-dbdb-479a-9bc3-7cfb5a061950 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.899322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3978c6-dbdb-479a-9bc3-7cfb5a061950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.899874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3978c6-dbdb-479a-9bc3-7cfb5a061950 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.901525 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3978c6-dbdb-479a-9bc3-7cfb5a061950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:59.905314 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3978c6-dbdb-479a-9bc3-7cfb5a061950 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:59.916219 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 805/1613] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:59.918223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c275-a6fe-4647-b96f-d5d304f94688 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.918507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c275-a6fe-4647-b96f-d5d304f94688 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.918876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c275-a6fe-4647-b96f-d5d304f94688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:59.930313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 809/1614] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:59.937689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3978c6-dbdb-479a-9bc3-7cfb5a061950 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:59.957197 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-39f6c275-a6fe-4647-b96f-d5d304f94688 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:59.959818 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-39f6c275-a6fe-4647-b96f-d5d304f94688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:59.971553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3978c6-dbdb-479a-9bc3-7cfb5a061950 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:59.978580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 806/1615] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:59.978580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6c275-a6fe-4647-b96f-d5d304f94688 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:59.986169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084449ab-5657-443c-92c2-11b79741fc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:59.987649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084449ab-5657-443c-92c2-11b79741fc94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:59.988361 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084449ab-5657-443c-92c2-11b79741fc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:59.988873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084449ab-5657-443c-92c2-11b79741fc94 admin admin] PATH_INFO: `/v3/projects/aa5fb93320db4b1a838bed8b14ca5653/users/a1b3f34373414c61b94e2a8d9018cbd5/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:00.000917 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6c275-a6fe-4647-b96f-d5d304f94688 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:00.009760 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f177c5c-96d9-41cd-9c33-0d333a785713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:00.011287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f177c5c-96d9-41cd-9c33-0d333a785713 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:00.011631 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f177c5c-96d9-41cd-9c33-0d333a785713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:00.011808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084449ab-5657-443c-92c2-11b79741fc94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa5fb93320db4b1a838bed8b14ca5653, user_id=a1b3f34373414c61b94e2a8d9018cbd5, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:00.012372 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f177c5c-96d9-41cd-9c33-0d333a785713 admin admin] PATH_INFO: `/v3/projects/8a813b391ab84dd686adb175da862e95/users/e840178a297a4ea28aed82a9ff9f40ad/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:00.016447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084449ab-5657-443c-92c2-11b79741fc94 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:00.037796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 810/1616] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:59 2022] PUT /identity/v3/projects/aa5fb93320db4b1a838bed8b14ca5653/users/a1b3f34373414c61b94e2a8d9018cbd5/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:00.038918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084449ab-5657-443c-92c2-11b79741fc94 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:00.045591 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f177c5c-96d9-41cd-9c33-0d333a785713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a813b391ab84dd686adb175da862e95, user_id=e840178a297a4ea28aed82a9ff9f40ad, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:00.051381 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f177c5c-96d9-41cd-9c33-0d333a785713 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:00.079639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 807/1617] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:59 2022] PUT /identity/v3/projects/8a813b391ab84dd686adb175da862e95/users/e840178a297a4ea28aed82a9ff9f40ad/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:00.081932 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cc00b-a9c6-4c19-8960-2636a7ff03f5 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:00.082710 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cc00b-a9c6-4c19-8960-2636a7ff03f5 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:00.083435 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cc00b-a9c6-4c19-8960-2636a7ff03f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:00.141052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-c52cc00b-a9c6-4c19-8960-2636a7ff03f5 None None] MFA Rules not processed for user `737e843d22c84e668b03e1437921cf05`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:00.144140 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084449ab-5657-443c-92c2-11b79741fc94 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:00.144728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c52cc00b-a9c6-4c19-8960-2636a7ff03f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:00.178843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:00.248752 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:00.284645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aca57e-5d60-4b28-b39d-d4713ff3eec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:00.285658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aca57e-5d60-4b28-b39d-d4713ff3eec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:00.286047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aca57e-5d60-4b28-b39d-d4713ff3eec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:00.286489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aca57e-5d60-4b28-b39d-d4713ff3eec5 admin admin] PATH_INFO: `/v3/projects/43b22599d66943c9919661ab4cd4a010/users/23deaaf412e64ba1bf53ed9e3a58fc70/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:00.289363 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 808/1618] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:00.291152 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cc00b-a9c6-4c19-8960-2636a7ff03f5 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:00.313531 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cc00b-a9c6-4c19-8960-2636a7ff03f5 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:00.316170 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aca57e-5d60-4b28-b39d-d4713ff3eec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43b22599d66943c9919661ab4cd4a010, user_id=23deaaf412e64ba1bf53ed9e3a58fc70, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:00.319109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aca57e-5d60-4b28-b39d-d4713ff3eec5 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:00.320893 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed46bb7-c4f2-4448-8ff9-1fa9ecfcbbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:00.321953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed46bb7-c4f2-4448-8ff9-1fa9ecfcbbb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:00.322402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed46bb7-c4f2-4448-8ff9-1fa9ecfcbbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:00.323005 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed46bb7-c4f2-4448-8ff9-1fa9ecfcbbb0 admin admin] PATH_INFO: `/v3/projects/144a5ee24aa44d4392da313cd405c715/users/2ba9fc062a1745eabc289b5586517b48/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:00.326751 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed46bb7-c4f2-4448-8ff9-1fa9ecfcbbb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=144a5ee24aa44d4392da313cd405c715, user_id=2ba9fc062a1745eabc289b5586517b48, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:00.331435 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed46bb7-c4f2-4448-8ff9-1fa9ecfcbbb0 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:00.344206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 811/1619] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:00 2022] PUT /identity/v3/projects/43b22599d66943c9919661ab4cd4a010/users/23deaaf412e64ba1bf53ed9e3a58fc70/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:00.345729 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aca57e-5d60-4b28-b39d-d4713ff3eec5 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:00.350369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 809/1620] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:00 2022] PUT /identity/v3/projects/144a5ee24aa44d4392da313cd405c715/users/2ba9fc062a1745eabc289b5586517b48/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:00.351293 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed46bb7-c4f2-4448-8ff9-1fa9ecfcbbb0 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:00.464019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aca57e-5d60-4b28-b39d-d4713ff3eec5 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:00.471400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b17c4-3846-4f72-9804-0770870c61c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:00.472584 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b17c4-3846-4f72-9804-0770870c61c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:00.472584 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b17c4-3846-4f72-9804-0770870c61c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:00.472584 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b17c4-3846-4f72-9804-0770870c61c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:00.475600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed46bb7-c4f2-4448-8ff9-1fa9ecfcbbb0 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:00.485546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bf588-c49a-4547-971d-de687f633816 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:00.487403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bf588-c49a-4547-971d-de687f633816 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:00.488304 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bf588-c49a-4547-971d-de687f633816 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:00.489013 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bf588-c49a-4547-971d-de687f633816 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:00.593330 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b17c4-3846-4f72-9804-0770870c61c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:00.594935 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b17c4-3846-4f72-9804-0770870c61c8 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:00.612700 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 812/1621] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:00.619361 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653eec1e-aca3-4870-a710-f37be8027908 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:00.620032 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653eec1e-aca3-4870-a710-f37be8027908 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:00.620843 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653eec1e-aca3-4870-a710-f37be8027908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:00.641598 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bf588-c49a-4547-971d-de687f633816 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:00.652501 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bf588-c49a-4547-971d-de687f633816 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:00.664303 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-653eec1e-aca3-4870-a710-f37be8027908 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:00.666423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-653eec1e-aca3-4870-a710-f37be8027908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:00.675623 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 810/1622] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:00.677191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bf588-c49a-4547-971d-de687f633816 service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:00.684032 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 813/1623] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:00.684032 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653eec1e-aca3-4870-a710-f37be8027908 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:00.691005 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bf588-c49a-4547-971d-de687f633816 service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:00.700663 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76652b-d783-4468-9836-f7a9acfff9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:00.701313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76652b-d783-4468-9836-f7a9acfff9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:00.701598 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76652b-d783-4468-9836-f7a9acfff9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:00.702086 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76652b-d783-4468-9836-f7a9acfff9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:00.702878 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76652b-d783-4468-9836-f7a9acfff9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:00.706513 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76652b-d783-4468-9836-f7a9acfff9c5 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:00.730223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 811/1624] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:00.734379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76889d-6c11-4c7f-a1bd-ccb851fc36fe None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:00.734991 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76889d-6c11-4c7f-a1bd-ccb851fc36fe None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:00.735524 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76889d-6c11-4c7f-a1bd-ccb851fc36fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:00.830058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-0e76889d-6c11-4c7f-a1bd-ccb851fc36fe None None] MFA Rules not processed for user `23deaaf412e64ba1bf53ed9e3a58fc70`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:00.832489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0e76889d-6c11-4c7f-a1bd-ccb851fc36fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:00.842407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653eec1e-aca3-4870-a710-f37be8027908 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:00.852716 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1535b-a47d-496d-8b43-72f8375471c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:00.853486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1535b-a47d-496d-8b43-72f8375471c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:00.854007 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1535b-a47d-496d-8b43-72f8375471c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:00.854315 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1535b-a47d-496d-8b43-72f8375471c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:00.923569 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:00.925947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:00.972590 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1535b-a47d-496d-8b43-72f8375471c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:00.972590 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1535b-a47d-496d-8b43-72f8375471c2 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:00.997700 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 814/1625] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:01.016193 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdc2f3-54be-4283-87ca-50583359d26a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:01.016499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdc2f3-54be-4283-87ca-50583359d26a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:01.016729 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdc2f3-54be-4283-87ca-50583359d26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:01.059942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 812/1626] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:01.063367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76889d-6c11-4c7f-a1bd-ccb851fc36fe None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:01.065677 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0e76889d-6c11-4c7f-a1bd-ccb851fc36fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:01.109624 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-3dfdc2f3-54be-4283-87ca-50583359d26a None None] MFA Rules not processed for user `2ba9fc062a1745eabc289b5586517b48`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:01.114009 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76889d-6c11-4c7f-a1bd-ccb851fc36fe None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:01.125932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3dfdc2f3-54be-4283-87ca-50583359d26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:01.139633 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a42b9e-c72d-4106-bd7c-740d66a66dd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:01.143536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a42b9e-c72d-4106-bd7c-740d66a66dd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:01.147084 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a42b9e-c72d-4106-bd7c-740d66a66dd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:01.149415 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a42b9e-c72d-4106-bd7c-740d66a66dd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:01.212499 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a42b9e-c72d-4106-bd7c-740d66a66dd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:01.217965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a42b9e-c72d-4106-bd7c-740d66a66dd6 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:01.263424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 813/1627] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:01.265502 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a42b9e-c72d-4106-bd7c-740d66a66dd6 service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:01.288981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a42b9e-c72d-4106-bd7c-740d66a66dd6 service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:01.300746 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580139da-9abe-4425-8abe-c71d18cff0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:01.301645 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580139da-9abe-4425-8abe-c71d18cff0a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:01.301924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580139da-9abe-4425-8abe-c71d18cff0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:01.302239 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580139da-9abe-4425-8abe-c71d18cff0a6 admin admin] PATH_INFO: `/v3/projects/8a813b391ab84dd686adb175da862e95/users/e840178a297a4ea28aed82a9ff9f40ad/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:01.304843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580139da-9abe-4425-8abe-c71d18cff0a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a813b391ab84dd686adb175da862e95, user_id=e840178a297a4ea28aed82a9ff9f40ad, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:01.308941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580139da-9abe-4425-8abe-c71d18cff0a6 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:01.340829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 814/1628] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:01 2022] PUT /identity/v3/projects/8a813b391ab84dd686adb175da862e95/users/e840178a297a4ea28aed82a9ff9f40ad/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:01.344565 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a49df-186a-456d-929d-c563c7c9c048 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:01.345019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a49df-186a-456d-929d-c563c7c9c048 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:01.345724 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a49df-186a-456d-929d-c563c7c9c048 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:01.349969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 815/1629] 10.222.0.25 () {54 vars in 962 bytes} [Tue Aug 30 14:44:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:44:01.355041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7b3c1-9f03-4350-822c-9d304b5e4021 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:01.356043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7b3c1-9f03-4350-822c-9d304b5e4021 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:01.356795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7b3c1-9f03-4350-822c-9d304b5e4021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:01.435909 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 815/1630] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:01.438852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195ff2a-658c-417d-9a32-48b128bde660 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:01.439152 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195ff2a-658c-417d-9a32-48b128bde660 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:01.439440 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195ff2a-658c-417d-9a32-48b128bde660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:01.471628 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-2bd7b3c1-9f03-4350-822c-9d304b5e4021 None None] MFA Rules not processed for user `e840178a297a4ea28aed82a9ff9f40ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:01.475198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2bd7b3c1-9f03-4350-822c-9d304b5e4021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:01.576384 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0195ff2a-658c-417d-9a32-48b128bde660 None None] MFA Rules not processed for user `cc95bbdafb0c4f34ad0662bc2e3ba4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:01.578357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0195ff2a-658c-417d-9a32-48b128bde660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:01.665242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 816/1631] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:01.665242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7b3c1-9f03-4350-822c-9d304b5e4021 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:01.778516 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7b3c1-9f03-4350-822c-9d304b5e4021 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:01.790635 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea88cf-68c8-4d9c-aadd-c00445f684ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:01.791324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea88cf-68c8-4d9c-aadd-c00445f684ec service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:01.792994 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea88cf-68c8-4d9c-aadd-c00445f684ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:01.793215 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea88cf-68c8-4d9c-aadd-c00445f684ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:01.817216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 816/1632] 10.222.0.25 () {58 vars in 1059 bytes} [Tue Aug 30 14:44:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:01.831886 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195ff2a-658c-417d-9a32-48b128bde660 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:01.858558 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195ff2a-658c-417d-9a32-48b128bde660 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:01.882843 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5eb9f7-cbff-43a5-bcff-f54e27400408 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:01.883705 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5eb9f7-cbff-43a5-bcff-f54e27400408 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:01.884674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5eb9f7-cbff-43a5-bcff-f54e27400408 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:01.886168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5eb9f7-cbff-43a5-bcff-f54e27400408 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:01.977250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:02.013389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:02.038226 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea88cf-68c8-4d9c-aadd-c00445f684ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:02.042961 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea88cf-68c8-4d9c-aadd-c00445f684ec service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:02.075788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 817/1633] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:02.117446 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5eb9f7-cbff-43a5-bcff-f54e27400408 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:02.119578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5eb9f7-cbff-43a5-bcff-f54e27400408 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:02.162591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 817/1634] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:44:02.226464 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea88cf-68c8-4d9c-aadd-c00445f684ec service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:02.260149 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea88cf-68c8-4d9c-aadd-c00445f684ec service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:02.281349 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363ee89-23a8-4169-9ab2-3f571de74d9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:02.287567 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363ee89-23a8-4169-9ab2-3f571de74d9b service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:02.288756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363ee89-23a8-4169-9ab2-3f571de74d9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:02.293726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363ee89-23a8-4169-9ab2-3f571de74d9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:02.342089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363ee89-23a8-4169-9ab2-3f571de74d9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:02.350969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363ee89-23a8-4169-9ab2-3f571de74d9b service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:02.399879 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 818/1635] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:02.767414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5eb9f7-cbff-43a5-bcff-f54e27400408 service nova] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:02.791189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5eb9f7-cbff-43a5-bcff-f54e27400408 service nova] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:02.807123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae1e9d-a769-40a6-a03b-6a6b8caf4dfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:02.808431 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae1e9d-a769-40a6-a03b-6a6b8caf4dfc service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:02.814382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae1e9d-a769-40a6-a03b-6a6b8caf4dfc service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:02.814879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae1e9d-a769-40a6-a03b-6a6b8caf4dfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:02.836453 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fdad2-8d11-493b-ba44-a22e5ce78ad9 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:02.837054 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fdad2-8d11-493b-ba44-a22e5ce78ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:02.837428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fdad2-8d11-493b-ba44-a22e5ce78ad9 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:02.843613 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 819/1636] 10.222.0.25 () {54 vars in 912 bytes} [Tue Aug 30 14:44:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:44:02.853082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2c3b4-f50b-4881-81d4-8a8911e1fc08 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:02.853404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2c3b4-f50b-4881-81d4-8a8911e1fc08 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:02.853908 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2c3b4-f50b-4881-81d4-8a8911e1fc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:02.961404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae1e9d-a769-40a6-a03b-6a6b8caf4dfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:02.961404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae1e9d-a769-40a6-a03b-6a6b8caf4dfc service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:02.971135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 818/1637] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:02.996775 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a3f2c3b4-f50b-4881-81d4-8a8911e1fc08 None None] MFA Rules not processed for user `cc95bbdafb0c4f34ad0662bc2e3ba4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:03.003639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a3f2c3b4-f50b-4881-81d4-8a8911e1fc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:03.057114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 820/1638] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Aug 30 14:44:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:44:03.191336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae1e9d-a769-40a6-a03b-6a6b8caf4dfc service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:03.220779 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae1e9d-a769-40a6-a03b-6a6b8caf4dfc service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:03.228417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5039e40-8ba2-4bed-8589-eb503baf4b5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:03.229403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5039e40-8ba2-4bed-8589-eb503baf4b5f service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:03.229665 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5039e40-8ba2-4bed-8589-eb503baf4b5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:03.229952 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5039e40-8ba2-4bed-8589-eb503baf4b5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:03.256336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5039e40-8ba2-4bed-8589-eb503baf4b5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:03.259221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5039e40-8ba2-4bed-8589-eb503baf4b5f service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:03.283059 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 819/1639] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:44:05.151239 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a9d40-f484-4ee7-80fc-c7b01b79794c None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:05.152230 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a9d40-f484-4ee7-80fc-c7b01b79794c None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:05.152230 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a9d40-f484-4ee7-80fc-c7b01b79794c None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:05.153730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 821/1640] 10.222.0.25 () {54 vars in 962 bytes} [Tue Aug 30 14:44:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:44:05.173933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13509ea6-5f8f-4311-a644-6e0095944481 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:05.174838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13509ea6-5f8f-4311-a644-6e0095944481 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:05.174838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13509ea6-5f8f-4311-a644-6e0095944481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:05.292501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-13509ea6-5f8f-4311-a644-6e0095944481 None None] MFA Rules not processed for user `cc95bbdafb0c4f34ad0662bc2e3ba4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:05.294585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-13509ea6-5f8f-4311-a644-6e0095944481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:05.356570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 820/1641] 10.222.0.25 () {58 vars in 1059 bytes} [Tue Aug 30 14:44:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:05.367697 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baedff4-d80d-43ab-9354-7dae295fc9e1 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:05.368631 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baedff4-d80d-43ab-9354-7dae295fc9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:05.369311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baedff4-d80d-43ab-9354-7dae295fc9e1 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:05.372846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 822/1642] 10.222.0.25 () {54 vars in 912 bytes} [Tue Aug 30 14:44:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:44:05.412558 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e03a0-8140-41ec-b0f4-396a068a6426 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:05.413769 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e03a0-8140-41ec-b0f4-396a068a6426 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:05.414947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e03a0-8140-41ec-b0f4-396a068a6426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:05.531725 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ca4e03a0-8140-41ec-b0f4-396a068a6426 None None] MFA Rules not processed for user `cc95bbdafb0c4f34ad0662bc2e3ba4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:05.537918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ca4e03a0-8140-41ec-b0f4-396a068a6426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:05.587368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 821/1643] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Aug 30 14:44:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:05.795941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baedff4-d80d-43ab-9354-7dae295fc9e1 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:05.871962 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baedff4-d80d-43ab-9354-7dae295fc9e1 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:05.871962 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73118c08-27fa-4c12-8ef9-fa3fe3f0393c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:05.871962 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73118c08-27fa-4c12-8ef9-fa3fe3f0393c service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:05.871962 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73118c08-27fa-4c12-8ef9-fa3fe3f0393c service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:05.872709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73118c08-27fa-4c12-8ef9-fa3fe3f0393c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:05.882501 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73118c08-27fa-4c12-8ef9-fa3fe3f0393c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:05.886676 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73118c08-27fa-4c12-8ef9-fa3fe3f0393c service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:05.913004 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 823/1644] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:13.709405 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e03a0-8140-41ec-b0f4-396a068a6426 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:13.787281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e03a0-8140-41ec-b0f4-396a068a6426 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:13.808592 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42399e-0361-4e0f-b32c-9cb332016494 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:13.810042 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42399e-0361-4e0f-b32c-9cb332016494 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:13.810771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42399e-0361-4e0f-b32c-9cb332016494 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:13.811582 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42399e-0361-4e0f-b32c-9cb332016494 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:14.130013 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42399e-0361-4e0f-b32c-9cb332016494 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:14.131901 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42399e-0361-4e0f-b32c-9cb332016494 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:14.156872 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 822/1645] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:14.310362 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2273881-0468-45c5-a59b-91c61376af49 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:14.310963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2273881-0468-45c5-a59b-91c61376af49 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:14.311262 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2273881-0468-45c5-a59b-91c61376af49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:14.470809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-e2273881-0468-45c5-a59b-91c61376af49 None None] MFA Rules not processed for user `521cc5ea8fb045bc81e8e3bfa8ba239b`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:14.475442 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e2273881-0468-45c5-a59b-91c61376af49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:14.785249 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 824/1646] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:14.814114 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42399e-0361-4e0f-b32c-9cb332016494 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:14.878960 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42399e-0361-4e0f-b32c-9cb332016494 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:14.932749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc89967-d484-47fa-a36b-54d74737a436 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:14.935504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc89967-d484-47fa-a36b-54d74737a436 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:14.942263 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc89967-d484-47fa-a36b-54d74737a436 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:14.943004 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc89967-d484-47fa-a36b-54d74737a436 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:15.035343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc89967-d484-47fa-a36b-54d74737a436 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:15.038102 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc89967-d484-47fa-a36b-54d74737a436 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:15.106010 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 823/1647] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:17.088616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785eab2-6cdd-436f-b87e-5666b492ecf0 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:17.089351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785eab2-6cdd-436f-b87e-5666b492ecf0 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:17.089557 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785eab2-6cdd-436f-b87e-5666b492ecf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:17.185329 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-c785eab2-6cdd-436f-b87e-5666b492ecf0 None None] MFA Rules not processed for user `3398704b0dbc4af08e94b774a88851a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:17.189576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c785eab2-6cdd-436f-b87e-5666b492ecf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:17.500906 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 825/1648] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:17.510705 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a5795-90ef-4d72-b6bd-d70c6fbb5e34 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:17.511449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a5795-90ef-4d72-b6bd-d70c6fbb5e34 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:17.511590 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a5795-90ef-4d72-b6bd-d70c6fbb5e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:17.601350 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c60a5795-90ef-4d72-b6bd-d70c6fbb5e34 None None] MFA Rules not processed for user `3398704b0dbc4af08e94b774a88851a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:17.603576 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c60a5795-90ef-4d72-b6bd-d70c6fbb5e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:17.638530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 824/1649] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:17.664136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2feaf20-193b-461a-b5f7-e66cd09d1c5d None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:17.665126 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2feaf20-193b-461a-b5f7-e66cd09d1c5d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:17.665709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2feaf20-193b-461a-b5f7-e66cd09d1c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:17.671001 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a5795-90ef-4d72-b6bd-d70c6fbb5e34 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:17.694416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a5795-90ef-4d72-b6bd-d70c6fbb5e34 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:17.703529 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cd2bf-6136-4e0d-b145-7869c187aeba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:17.704629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cd2bf-6136-4e0d-b145-7869c187aeba service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:17.705169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cd2bf-6136-4e0d-b145-7869c187aeba service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:17.705667 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cd2bf-6136-4e0d-b145-7869c187aeba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:17.732751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cd2bf-6136-4e0d-b145-7869c187aeba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:17.735735 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cd2bf-6136-4e0d-b145-7869c187aeba service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:17.769534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 825/1650] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:44:17.888425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a2feaf20-193b-461a-b5f7-e66cd09d1c5d None None] MFA Rules not processed for user `a1b3f34373414c61b94e2a8d9018cbd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:17.908414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a2feaf20-193b-461a-b5f7-e66cd09d1c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:18.034238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:18.308994 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 826/1651] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 646 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:18.312430 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cd2bf-6136-4e0d-b145-7869c187aeba service nova] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:18.338094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:18.350052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cd2bf-6136-4e0d-b145-7869c187aeba service nova] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:18.382134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89116933-5361-4535-afd9-795f4975bba0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:18.382875 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89116933-5361-4535-afd9-795f4975bba0 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:18.384517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89116933-5361-4535-afd9-795f4975bba0 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:18.384826 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89116933-5361-4535-afd9-795f4975bba0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:18.446555 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89116933-5361-4535-afd9-795f4975bba0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:18.451229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89116933-5361-4535-afd9-795f4975bba0 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:18.494160 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 826/1652] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:19.206007 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15f53a-e1da-414b-9086-c1a142425a1b None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:19.207216 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15f53a-e1da-414b-9086-c1a142425a1b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:19.208131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15f53a-e1da-414b-9086-c1a142425a1b None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:19.212321 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 827/1653] 10.222.0.25 () {54 vars in 912 bytes} [Tue Aug 30 14:44:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:44:19.219343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7099e-4dff-47f7-8596-cde738f30d5c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:19.220490 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7099e-4dff-47f7-8596-cde738f30d5c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:19.221140 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7099e-4dff-47f7-8596-cde738f30d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:19.308312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-04b7099e-4dff-47f7-8596-cde738f30d5c None None] MFA Rules not processed for user `932fce741c5a4deba79a310a14e6bd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:19.319810 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-04b7099e-4dff-47f7-8596-cde738f30d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:19.504101 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 827/1654] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Aug 30 14:44:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:44:19.609299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15f53a-e1da-414b-9086-c1a142425a1b None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:19.635478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15f53a-e1da-414b-9086-c1a142425a1b None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:19.651793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4926a-9ecb-4ec4-90fb-cc0b6bb639bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:19.652473 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4926a-9ecb-4ec4-90fb-cc0b6bb639bf service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:19.652726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4926a-9ecb-4ec4-90fb-cc0b6bb639bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:19.656210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4926a-9ecb-4ec4-90fb-cc0b6bb639bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:19.684987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4926a-9ecb-4ec4-90fb-cc0b6bb639bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:19.696508 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4926a-9ecb-4ec4-90fb-cc0b6bb639bf service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:19.733275 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 828/1655] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:44:21.036273 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7099e-4dff-47f7-8596-cde738f30d5c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:21.249540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7997e-de6c-4d1e-a4d3-8d67189e089a None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:21.250903 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7997e-de6c-4d1e-a4d3-8d67189e089a None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:21.251428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7997e-de6c-4d1e-a4d3-8d67189e089a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:21.270702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7099e-4dff-47f7-8596-cde738f30d5c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:21.308089 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2f522-539b-40de-aad6-8834bf2c7fb9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cdd6b1ee9f4e97beb9bb037e446c8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:21.309616 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2f522-539b-40de-aad6-8834bf2c7fb9 service placement] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:21.310281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2f522-539b-40de-aad6-8834bf2c7fb9 service placement] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:21.310947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2f522-539b-40de-aad6-8834bf2c7fb9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:21.319628 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ecf2f522-539b-40de-aad6-8834bf2c7fb9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:21.321136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-e9a7997e-de6c-4d1e-a4d3-8d67189e089a None None] MFA Rules not processed for user `3398704b0dbc4af08e94b774a88851a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:21.323125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e9a7997e-de6c-4d1e-a4d3-8d67189e089a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:21.366315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 829/1656] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:44:21.382274 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2f522-539b-40de-aad6-8834bf2c7fb9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:21.385755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2f522-539b-40de-aad6-8834bf2c7fb9 service placement] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:21.436317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 828/1657] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:22.009702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203c586-9e90-4663-8113-bb28c553bccc None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:22.010877 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203c586-9e90-4663-8113-bb28c553bccc None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:22.011175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203c586-9e90-4663-8113-bb28c553bccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:22.084100 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-2203c586-9e90-4663-8113-bb28c553bccc None None] MFA Rules not processed for user `ee02494e189d4823ab776ea3da9fd2ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:22.085822 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2203c586-9e90-4663-8113-bb28c553bccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:22.185722 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 830/1658] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:22.197880 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e4640-187d-4698-bf6d-495fe5cf689c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:22.198509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e4640-187d-4698-bf6d-495fe5cf689c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:22.198778 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e4640-187d-4698-bf6d-495fe5cf689c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:22.206699 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbd882-f038-4001-8a12-e0e15cc5d3c8 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:22.207064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbd882-f038-4001-8a12-e0e15cc5d3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:22.207259 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbd882-f038-4001-8a12-e0e15cc5d3c8 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:22.209067 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 831/1659] 10.222.0.25 () {54 vars in 912 bytes} [Tue Aug 30 14:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:44:22.216132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8208e33-f571-45c2-b93c-ad058bfdc739 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:22.216465 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8208e33-f571-45c2-b93c-ad058bfdc739 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:22.216746 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8208e33-f571-45c2-b93c-ad058bfdc739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:22.238976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d58e4640-187d-4698-bf6d-495fe5cf689c None None] MFA Rules not processed for user `ee02494e189d4823ab776ea3da9fd2ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:22.243177 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d58e4640-187d-4698-bf6d-495fe5cf689c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:22.259756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-f8208e33-f571-45c2-b93c-ad058bfdc739 None None] MFA Rules not processed for user `932fce741c5a4deba79a310a14e6bd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:22.261626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 829/1660] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:22.271048 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f8208e33-f571-45c2-b93c-ad058bfdc739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:22.287009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e4640-187d-4698-bf6d-495fe5cf689c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:22.295606 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 832/1661] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Aug 30 14:44:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:44:22.308646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e4640-187d-4698-bf6d-495fe5cf689c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:22.319346 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e954cfc-9cc5-488f-a412-eebad2784232 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:22.321012 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e954cfc-9cc5-488f-a412-eebad2784232 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:22.321883 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e954cfc-9cc5-488f-a412-eebad2784232 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:22.323268 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e954cfc-9cc5-488f-a412-eebad2784232 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:22.343862 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e954cfc-9cc5-488f-a412-eebad2784232 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:22.346317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e954cfc-9cc5-488f-a412-eebad2784232 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:22.360661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 830/1662] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:44:22.380351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8208e33-f571-45c2-b93c-ad058bfdc739 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:22.396982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8208e33-f571-45c2-b93c-ad058bfdc739 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:22.413164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904c477-8670-4e90-a9de-77b5ec11587a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:22.413912 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904c477-8670-4e90-a9de-77b5ec11587a service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:22.414244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904c477-8670-4e90-a9de-77b5ec11587a service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:22.414621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904c477-8670-4e90-a9de-77b5ec11587a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:22.434748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4904c477-8670-4e90-a9de-77b5ec11587a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:22.441027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4904c477-8670-4e90-a9de-77b5ec11587a service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:22.466582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 833/1663] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:22.573403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0859a9-b43f-44b2-88a0-6c03cc5fee63 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:22.573403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0859a9-b43f-44b2-88a0-6c03cc5fee63 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:22.573403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0859a9-b43f-44b2-88a0-6c03cc5fee63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:22.642935 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0d0859a9-b43f-44b2-88a0-6c03cc5fee63 None None] MFA Rules not processed for user `521cc5ea8fb045bc81e8e3bfa8ba239b`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:22.645320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0d0859a9-b43f-44b2-88a0-6c03cc5fee63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:22.662523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 831/1664] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:25.239574 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69472dce-4c80-48c1-ae22-3b60c0007408 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:25.240541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69472dce-4c80-48c1-ae22-3b60c0007408 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:25.241314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69472dce-4c80-48c1-ae22-3b60c0007408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:25.327505 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-69472dce-4c80-48c1-ae22-3b60c0007408 None None] MFA Rules not processed for user `a1b3f34373414c61b94e2a8d9018cbd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:25.329801 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-69472dce-4c80-48c1-ae22-3b60c0007408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:25.389468 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 834/1665] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:44:30.742922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0859a9-b43f-44b2-88a0-6c03cc5fee63 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:31.049072 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:31.091091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0859a9-b43f-44b2-88a0-6c03cc5fee63 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:31.115429 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f0463-c921-43d9-b5fd-d5b67b2fbddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:31.116681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f0463-c921-43d9-b5fd-d5b67b2fbddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:31.118180 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f0463-c921-43d9-b5fd-d5b67b2fbddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:31.118740 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f0463-c921-43d9-b5fd-d5b67b2fbddb admin admin] PATH_INFO: `/v3/users/23deaaf412e64ba1bf53ed9e3a58fc70` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:31.129157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f0463-c921-43d9-b5fd-d5b67b2fbddb admin admin] RBAC: Authorizing `identity:delete_user(user_id=23deaaf412e64ba1bf53ed9e3a58fc70)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:31.141054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f0463-c921-43d9-b5fd-d5b67b2fbddb admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:31.149693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75a746-3680-4cf0-ab78-a70419d99801 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:31.152452 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75a746-3680-4cf0-ab78-a70419d99801 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:31.153047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75a746-3680-4cf0-ab78-a70419d99801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:31.375320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-479f0463-c921-43d9-b5fd-d5b67b2fbddb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23deaaf412e64ba1bf53ed9e3a58fc70'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:31.378818 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-0d75a746-3680-4cf0-ab78-a70419d99801 None None] MFA Rules not processed for user `8fc940594a1848aa9644b645049a1bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:31.383280 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0d75a746-3680-4cf0-ab78-a70419d99801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:31.408125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-479f0463-c921-43d9-b5fd-d5b67b2fbddb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23deaaf412e64ba1bf53ed9e3a58fc70'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:31.438486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-479f0463-c921-43d9-b5fd-d5b67b2fbddb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23deaaf412e64ba1bf53ed9e3a58fc70'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:31.439921 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-479f0463-c921-43d9-b5fd-d5b67b2fbddb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23deaaf412e64ba1bf53ed9e3a58fc70'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:31.547102 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-479f0463-c921-43d9-b5fd-d5b67b2fbddb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23deaaf412e64ba1bf53ed9e3a58fc70'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:31.557074 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:31.579456 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 832/1666] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:30 2022] DELETE /identity/v3/users/23deaaf412e64ba1bf53ed9e3a58fc70 => generated 0 bytes in 834 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:44:31.585263 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8d18b-9ddf-4754-b39f-854f9c07bf4b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:31.585709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8d18b-9ddf-4754-b39f-854f9c07bf4b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:31.585993 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8d18b-9ddf-4754-b39f-854f9c07bf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:31.661970 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d1c8d18b-9ddf-4754-b39f-854f9c07bf4b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:31.671874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d1c8d18b-9ddf-4754-b39f-854f9c07bf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:31.704908 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 835/1667] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Aug 30 14:44:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 559 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:44:31.966907 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 833/1668] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:31.995637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75a746-3680-4cf0-ab78-a70419d99801 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:31.997302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8d18b-9ddf-4754-b39f-854f9c07bf4b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:31.999088 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0d75a746-3680-4cf0-ab78-a70419d99801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:31.999731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d1c8d18b-9ddf-4754-b39f-854f9c07bf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:32.044078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75a746-3680-4cf0-ab78-a70419d99801 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:32.323673 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122b524-b169-4a85-b89a-c9cb054a0ba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:32.328643 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b524-b169-4a85-b89a-c9cb054a0ba3 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:32.331084 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b524-b169-4a85-b89a-c9cb054a0ba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:32.334029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b524-b169-4a85-b89a-c9cb054a0ba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:32.361165 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:32.410854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122b524-b169-4a85-b89a-c9cb054a0ba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:32.414869 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8d18b-9ddf-4754-b39f-854f9c07bf4b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:32.415889 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122b524-b169-4a85-b89a-c9cb054a0ba3 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:32.425879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f276f48-cd2a-43e1-869a-9d47f1ad9680 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d98f5adfaf410580481417e7ba1638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:32.427145 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f276f48-cd2a-43e1-869a-9d47f1ad9680 service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:32.427664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f276f48-cd2a-43e1-869a-9d47f1ad9680 service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:32.428478 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f276f48-cd2a-43e1-869a-9d47f1ad9680 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:32.449744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 836/1669] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:32.716019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f276f48-cd2a-43e1-869a-9d47f1ad9680 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:32.720235 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f276f48-cd2a-43e1-869a-9d47f1ad9680 service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:32.776919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 834/1670] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Aug 30 14:44:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 781 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 14:44:33.479179 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122b524-b169-4a85-b89a-c9cb054a0ba3 service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:33.482576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b122b524-b169-4a85-b89a-c9cb054a0ba3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:33.518542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122b524-b169-4a85-b89a-c9cb054a0ba3 service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:33.538952 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1ecc8-7e50-4411-b6f8-982f4de4e572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:33.539920 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1ecc8-7e50-4411-b6f8-982f4de4e572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:33.541371 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1ecc8-7e50-4411-b6f8-982f4de4e572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:33.541371 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1ecc8-7e50-4411-b6f8-982f4de4e572 admin admin] PATH_INFO: `/v3/projects/43b22599d66943c9919661ab4cd4a010` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:33.543677 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1ecc8-7e50-4411-b6f8-982f4de4e572 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43b22599d66943c9919661ab4cd4a010)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:33.548955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1ecc8-7e50-4411-b6f8-982f4de4e572 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:33.673709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6df1ecc8-7e50-4411-b6f8-982f4de4e572 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43b22599d66943c9919661ab4cd4a010', 'request_id': 'req-6df1ecc8-7e50-4411-b6f8-982f4de4e572'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:33.683147 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6df1ecc8-7e50-4411-b6f8-982f4de4e572 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43b22599d66943c9919661ab4cd4a010', 'request_id': 'req-6df1ecc8-7e50-4411-b6f8-982f4de4e572'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:33.692602 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6df1ecc8-7e50-4411-b6f8-982f4de4e572 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43b22599d66943c9919661ab4cd4a010', 'request_id': 'req-6df1ecc8-7e50-4411-b6f8-982f4de4e572'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:33.723086 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6df1ecc8-7e50-4411-b6f8-982f4de4e572 admin admin] The token cache is being invalidate because project 43b22599d66943c9919661ab4cd4a010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:33.723756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6df1ecc8-7e50-4411-b6f8-982f4de4e572 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:33.734362 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6df1ecc8-7e50-4411-b6f8-982f4de4e572 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:33.742212 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 837/1671] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:33 2022] DELETE /identity/v3/projects/43b22599d66943c9919661ab4cd4a010 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:33.747309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f33b8-8241-4975-9601-58b6fff2f89b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:33.748302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f33b8-8241-4975-9601-58b6fff2f89b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:33.748778 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f33b8-8241-4975-9601-58b6fff2f89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:33.772966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-490f33b8-8241-4975-9601-58b6fff2f89b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:33.775043 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-490f33b8-8241-4975-9601-58b6fff2f89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:33.953256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 835/1672] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:33.955139 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ea3e5-5a3d-4304-beed-b44412e8d010 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:33.957749 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ea3e5-5a3d-4304-beed-b44412e8d010 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:33.958587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ea3e5-5a3d-4304-beed-b44412e8d010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:34.034231 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-396ea3e5-5a3d-4304-beed-b44412e8d010 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:34.036018 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-396ea3e5-5a3d-4304-beed-b44412e8d010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:34.063543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 838/1673] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:34.069215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f33b8-8241-4975-9601-58b6fff2f89b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:34.088279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f33b8-8241-4975-9601-58b6fff2f89b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:34.111258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf886d-fa42-4ed4-b65e-b752009f43a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:34.112815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf886d-fa42-4ed4-b65e-b752009f43a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:34.113528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf886d-fa42-4ed4-b65e-b752009f43a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:34.114165 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf886d-fa42-4ed4-b65e-b752009f43a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:34.115576 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf886d-fa42-4ed4-b65e-b752009f43a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:44:34.116463 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf886d-fa42-4ed4-b65e-b752009f43a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:34.122360 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf886d-fa42-4ed4-b65e-b752009f43a6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:34.148772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 836/1674] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:44:34.156585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ea3e5-5a3d-4304-beed-b44412e8d010 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:34.190248 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ea3e5-5a3d-4304-beed-b44412e8d010 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:34.201588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c528a19-3eeb-4bd3-8517-2b9a9e04cbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:34.202913 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c528a19-3eeb-4bd3-8517-2b9a9e04cbcc admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:34.203802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c528a19-3eeb-4bd3-8517-2b9a9e04cbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:34.207316 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c528a19-3eeb-4bd3-8517-2b9a9e04cbcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:34.208855 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c528a19-3eeb-4bd3-8517-2b9a9e04cbcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:34.213022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c528a19-3eeb-4bd3-8517-2b9a9e04cbcc admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:34.340490 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 839/1675] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:34 2022] POST /identity/v3/projects => generated 375 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:34.344993 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf886d-fa42-4ed4-b65e-b752009f43a6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:34.364672 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf886d-fa42-4ed4-b65e-b752009f43a6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:34.622847 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd7761-48aa-4b21-be41-169640c89022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:34.623670 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd7761-48aa-4b21-be41-169640c89022 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:34.625234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd7761-48aa-4b21-be41-169640c89022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:34.625546 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd7761-48aa-4b21-be41-169640c89022 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:34.628286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd7761-48aa-4b21-be41-169640c89022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:34.634607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd7761-48aa-4b21-be41-169640c89022 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:34.717660 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 837/1676] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:44:34 2022] POST /identity/v3/users => generated 288 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:34.720227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c528a19-3eeb-4bd3-8517-2b9a9e04cbcc admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:34.756115 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c528a19-3eeb-4bd3-8517-2b9a9e04cbcc admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:34.779431 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1f6ea-afcc-4654-809a-03dd7edc4e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:34.781896 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1f6ea-afcc-4654-809a-03dd7edc4e0b admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:34.782613 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1f6ea-afcc-4654-809a-03dd7edc4e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:34.783287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1f6ea-afcc-4654-809a-03dd7edc4e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:34.784779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1f6ea-afcc-4654-809a-03dd7edc4e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:34.801192 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1f6ea-afcc-4654-809a-03dd7edc4e0b admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:34.854979 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd7761-48aa-4b21-be41-169640c89022 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:34.862764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 840/1677] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:34.890608 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd7761-48aa-4b21-be41-169640c89022 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:34.921599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f2899-b0ff-4e26-9f1f-1687442dc825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:34.923466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f2899-b0ff-4e26-9f1f-1687442dc825 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:34.923677 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f2899-b0ff-4e26-9f1f-1687442dc825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:34.924102 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f2899-b0ff-4e26-9f1f-1687442dc825 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:34.925346 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f2899-b0ff-4e26-9f1f-1687442dc825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:34.928604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f2899-b0ff-4e26-9f1f-1687442dc825 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:34.939844 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:34.980722 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1f6ea-afcc-4654-809a-03dd7edc4e0b admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:34.981892 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 838/1678] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:35.058378 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1f6ea-afcc-4654-809a-03dd7edc4e0b admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:35.072669 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abda5ef-b14c-4669-ad10-c50ab542f410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:35.074380 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abda5ef-b14c-4669-ad10-c50ab542f410 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:35.075112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abda5ef-b14c-4669-ad10-c50ab542f410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:35.075778 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abda5ef-b14c-4669-ad10-c50ab542f410 admin admin] PATH_INFO: `/v3/projects/c9c3c148a4cc4f93a6b75a4fa10cae44/users/5b14af6fb579472593035910027a13f7/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:35.117417 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abda5ef-b14c-4669-ad10-c50ab542f410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c3c148a4cc4f93a6b75a4fa10cae44, user_id=5b14af6fb579472593035910027a13f7, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:35.122705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abda5ef-b14c-4669-ad10-c50ab542f410 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:35.142475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:35.150211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 841/1679] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:34 2022] PUT /identity/v3/projects/c9c3c148a4cc4f93a6b75a4fa10cae44/users/5b14af6fb579472593035910027a13f7/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:44:35.178019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7f0da-8e77-4720-823b-a40a819069c2 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:35.178431 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7f0da-8e77-4720-823b-a40a819069c2 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:35.178921 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7f0da-8e77-4720-823b-a40a819069c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:35.303957 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-38e7f0da-8e77-4720-823b-a40a819069c2 None None] MFA Rules not processed for user `5b14af6fb579472593035910027a13f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:35.305288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-38e7f0da-8e77-4720-823b-a40a819069c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:35.673241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64d09d-df61-4467-a693-a8fb7bbb6152 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:35.673512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64d09d-df61-4467-a693-a8fb7bbb6152 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:35.673855 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64d09d-df61-4467-a693-a8fb7bbb6152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:35.715338 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 839/1680] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 539 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:35.734858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7f0da-8e77-4720-823b-a40a819069c2 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:35.745441 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-38e7f0da-8e77-4720-823b-a40a819069c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:35.790427 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-cf64d09d-df61-4467-a693-a8fb7bbb6152 None None] MFA Rules not processed for user `932fce741c5a4deba79a310a14e6bd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:35.802313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-cf64d09d-df61-4467-a693-a8fb7bbb6152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:36.015232 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 842/1681] 10.222.0.57 () {58 vars in 1015 bytes} [Tue Aug 30 14:44:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:36.095765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7f0da-8e77-4720-823b-a40a819069c2 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:36.141614 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2647103-ddd7-4847-9964-b239cb6f347a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:36.142380 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2647103-ddd7-4847-9964-b239cb6f347a service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:36.142778 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2647103-ddd7-4847-9964-b239cb6f347a service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:36.143153 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2647103-ddd7-4847-9964-b239cb6f347a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:36.189521 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2647103-ddd7-4847-9964-b239cb6f347a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:36.191737 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2647103-ddd7-4847-9964-b239cb6f347a service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:36.227311 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 840/1682] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:44:36.676581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64d09d-df61-4467-a693-a8fb7bbb6152 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:36.679667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-cf64d09d-df61-4467-a693-a8fb7bbb6152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:36.682554 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2647103-ddd7-4847-9964-b239cb6f347a service nova] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:36.684207 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d2647103-ddd7-4847-9964-b239cb6f347a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:36.705159 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2647103-ddd7-4847-9964-b239cb6f347a service nova] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:36.724822 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491186d-21b3-4d70-b2c2-672b00ae4756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:36.732370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491186d-21b3-4d70-b2c2-672b00ae4756 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:36.734043 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491186d-21b3-4d70-b2c2-672b00ae4756 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:36.735293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491186d-21b3-4d70-b2c2-672b00ae4756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:36.755178 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64d09d-df61-4467-a693-a8fb7bbb6152 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:36.761069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491186d-21b3-4d70-b2c2-672b00ae4756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:36.786130 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491186d-21b3-4d70-b2c2-672b00ae4756 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:36.819840 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee529da-00bc-4b7b-aa17-c6151d989749 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:36.821091 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee529da-00bc-4b7b-aa17-c6151d989749 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:36.829956 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 841/1683] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:36.837255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491186d-21b3-4d70-b2c2-672b00ae4756 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:36.841591 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee529da-00bc-4b7b-aa17-c6151d989749 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:36.843221 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee529da-00bc-4b7b-aa17-c6151d989749 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:36.850805 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491186d-21b3-4d70-b2c2-672b00ae4756 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:36.868539 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745884f9-fed3-4588-a488-69ce514b1d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:36.869550 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745884f9-fed3-4588-a488-69ce514b1d75 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:36.870162 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745884f9-fed3-4588-a488-69ce514b1d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:36.870589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745884f9-fed3-4588-a488-69ce514b1d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:36.894369 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745884f9-fed3-4588-a488-69ce514b1d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:36.897143 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745884f9-fed3-4588-a488-69ce514b1d75 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:36.912673 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 842/1684] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:36.955459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee529da-00bc-4b7b-aa17-c6151d989749 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:36.970405 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee529da-00bc-4b7b-aa17-c6151d989749 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:37.040192 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 843/1685] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:37.827324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745884f9-fed3-4588-a488-69ce514b1d75 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:37.835540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-745884f9-fed3-4588-a488-69ce514b1d75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:38.063111 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:38.112868 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745884f9-fed3-4588-a488-69ce514b1d75 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:38.141158 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d0b39-388a-45a6-b385-3db3e214c80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:38.146738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d0b39-388a-45a6-b385-3db3e214c80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:38.148407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d0b39-388a-45a6-b385-3db3e214c80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:38.149265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d0b39-388a-45a6-b385-3db3e214c80a admin admin] PATH_INFO: `/v3/users/c875c46a88e04b839dee75daf10c156d` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:38.155285 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d0b39-388a-45a6-b385-3db3e214c80a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c875c46a88e04b839dee75daf10c156d)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:38.175007 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d0b39-388a-45a6-b385-3db3e214c80a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:38.438118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-591d0b39-388a-45a6-b385-3db3e214c80a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c875c46a88e04b839dee75daf10c156d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:38.500443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-591d0b39-388a-45a6-b385-3db3e214c80a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c875c46a88e04b839dee75daf10c156d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:38.554007 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-591d0b39-388a-45a6-b385-3db3e214c80a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c875c46a88e04b839dee75daf10c156d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:38.555168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-591d0b39-388a-45a6-b385-3db3e214c80a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c875c46a88e04b839dee75daf10c156d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:38.637220 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-591d0b39-388a-45a6-b385-3db3e214c80a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c875c46a88e04b839dee75daf10c156d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:38.676333 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 843/1686] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:37 2022] DELETE /identity/v3/users/c875c46a88e04b839dee75daf10c156d => generated 0 bytes in 847 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:38.680394 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52e4b9-9247-410b-9fcb-16b48e63121a None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:38.681015 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52e4b9-9247-410b-9fcb-16b48e63121a None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:38.681344 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52e4b9-9247-410b-9fcb-16b48e63121a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:38.773387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d0b39-388a-45a6-b385-3db3e214c80a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:38.775980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-8e52e4b9-9247-410b-9fcb-16b48e63121a None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:38.776936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-591d0b39-388a-45a6-b385-3db3e214c80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:38.778989 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-8e52e4b9-9247-410b-9fcb-16b48e63121a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:39.173097 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d0b39-388a-45a6-b385-3db3e214c80a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:39.210540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1a639-1035-47f1-ad5f-1eb5909a5a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:39.211607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1a639-1035-47f1-ad5f-1eb5909a5a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:39.212828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1a639-1035-47f1-ad5f-1eb5909a5a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:39.214065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1a639-1035-47f1-ad5f-1eb5909a5a84 admin admin] PATH_INFO: `/v3/users/5b14af6fb579472593035910027a13f7` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:39.217180 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1a639-1035-47f1-ad5f-1eb5909a5a84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b14af6fb579472593035910027a13f7)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:39.225390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1a639-1035-47f1-ad5f-1eb5909a5a84 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:39.269598 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 844/1687] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 588 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:39.367797 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52e4b9-9247-410b-9fcb-16b48e63121a None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:39.375116 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-8e52e4b9-9247-410b-9fcb-16b48e63121a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:39.408870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f5c1a639-1035-47f1-ad5f-1eb5909a5a84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b14af6fb579472593035910027a13f7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:39.426781 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f5c1a639-1035-47f1-ad5f-1eb5909a5a84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b14af6fb579472593035910027a13f7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:39.446228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f5c1a639-1035-47f1-ad5f-1eb5909a5a84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b14af6fb579472593035910027a13f7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:39.446982 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f5c1a639-1035-47f1-ad5f-1eb5909a5a84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b14af6fb579472593035910027a13f7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:39.454212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:39.477850 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f5c1a639-1035-47f1-ad5f-1eb5909a5a84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b14af6fb579472593035910027a13f7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:39.493814 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 844/1688] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:38 2022] DELETE /identity/v3/users/5b14af6fb579472593035910027a13f7 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:44:39.496283 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37df3e-3cb9-41bc-9782-dcde90179fa7 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:39.496539 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37df3e-3cb9-41bc-9782-dcde90179fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:39.496844 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37df3e-3cb9-41bc-9782-dcde90179fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:39.550875 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-df37df3e-3cb9-41bc-9782-dcde90179fa7 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:39.552212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-df37df3e-3cb9-41bc-9782-dcde90179fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:39.661062 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52e4b9-9247-410b-9fcb-16b48e63121a None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:39.800843 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 845/1689] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:39.807434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780d26a-3395-4282-91e4-ffa97bc51961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:39.810451 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780d26a-3395-4282-91e4-ffa97bc51961 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:39.811215 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780d26a-3395-4282-91e4-ffa97bc51961 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:39.811644 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780d26a-3395-4282-91e4-ffa97bc51961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:39.821721 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-1780d26a-3395-4282-91e4-ffa97bc51961 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:39.840695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:39.864415 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780d26a-3395-4282-91e4-ffa97bc51961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:39.866907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780d26a-3395-4282-91e4-ffa97bc51961 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:39.907627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 845/1690] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:40.204043 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37df3e-3cb9-41bc-9782-dcde90179fa7 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:40.214457 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-df37df3e-3cb9-41bc-9782-dcde90179fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:40.280188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780d26a-3395-4282-91e4-ffa97bc51961 service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:40.281963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-1780d26a-3395-4282-91e4-ffa97bc51961 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:40.287605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37df3e-3cb9-41bc-9782-dcde90179fa7 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:40.306282 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780d26a-3395-4282-91e4-ffa97bc51961 service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:40.309509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d6e67-7dfa-4caa-bde9-2b65f874fe2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:40.317004 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d6e67-7dfa-4caa-bde9-2b65f874fe2a service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:40.317786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d6e67-7dfa-4caa-bde9-2b65f874fe2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:40.318813 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d6e67-7dfa-4caa-bde9-2b65f874fe2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:40.321940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a26ca-ce4b-4a64-acee-899f6e94f743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:40.322674 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a26ca-ce4b-4a64-acee-899f6e94f743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:40.322900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a26ca-ce4b-4a64-acee-899f6e94f743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:40.323173 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a26ca-ce4b-4a64-acee-899f6e94f743 admin admin] PATH_INFO: `/v3/users/737e843d22c84e668b03e1437921cf05` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:40.324768 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a26ca-ce4b-4a64-acee-899f6e94f743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=737e843d22c84e668b03e1437921cf05)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:40.325915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: WARNING py.warnings [None req-415a26ca-ce4b-4a64-acee-899f6e94f743 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:44:40.325915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: warnings.warn(msg) Aug 30 14:44:40.325915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: Aug 30 14:44:40.329487 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a26ca-ce4b-4a64-acee-899f6e94f743 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:40.351609 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d6e67-7dfa-4caa-bde9-2b65f874fe2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:40.354760 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d6e67-7dfa-4caa-bde9-2b65f874fe2a service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:40.402534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 846/1691] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:44:40.580314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-415a26ca-ce4b-4a64-acee-899f6e94f743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '737e843d22c84e668b03e1437921cf05'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:40.589442 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-415a26ca-ce4b-4a64-acee-899f6e94f743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '737e843d22c84e668b03e1437921cf05'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:40.593948 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-415a26ca-ce4b-4a64-acee-899f6e94f743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '737e843d22c84e668b03e1437921cf05'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:40.623825 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-415a26ca-ce4b-4a64-acee-899f6e94f743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '737e843d22c84e668b03e1437921cf05'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:40.697612 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-415a26ca-ce4b-4a64-acee-899f6e94f743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '737e843d22c84e668b03e1437921cf05'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:40.724042 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 846/1692] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:40 2022] DELETE /identity/v3/users/737e843d22c84e668b03e1437921cf05 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:44:40.730243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc5990-f354-478b-aab2-8bd87cde937b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:40.730243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc5990-f354-478b-aab2-8bd87cde937b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:40.730243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc5990-f354-478b-aab2-8bd87cde937b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:40.791124 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e9bc5990-f354-478b-aab2-8bd87cde937b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:40.798413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e9bc5990-f354-478b-aab2-8bd87cde937b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:41.116078 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 847/1693] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:41.171655 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a26ca-ce4b-4a64-acee-899f6e94f743 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:41.173937 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-415a26ca-ce4b-4a64-acee-899f6e94f743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:41.422308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a26ca-ce4b-4a64-acee-899f6e94f743 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:41.432435 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab7c4f-36d5-436b-989b-73fb3d5205dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:41.433103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab7c4f-36d5-436b-989b-73fb3d5205dd service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:41.433459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab7c4f-36d5-436b-989b-73fb3d5205dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:41.433631 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab7c4f-36d5-436b-989b-73fb3d5205dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:41.475792 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab7c4f-36d5-436b-989b-73fb3d5205dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:41.477353 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab7c4f-36d5-436b-989b-73fb3d5205dd service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:41.519701 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 847/1694] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:41.986225 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc5990-f354-478b-aab2-8bd87cde937b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:41.989202 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e9bc5990-f354-478b-aab2-8bd87cde937b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:42.016622 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc5990-f354-478b-aab2-8bd87cde937b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:42.030595 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:42.031354 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:42.031652 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:42.034240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb admin admin] PATH_INFO: `/v3/projects/c9c3c148a4cc4f93a6b75a4fa10cae44` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:42.036607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9c3c148a4cc4f93a6b75a4fa10cae44)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:42.037702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: WARNING py.warnings [None req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:44:42.037702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: warnings.warn(msg) Aug 30 14:44:42.037702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: Aug 30 14:44:42.038956 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:42.194243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9c3c148a4cc4f93a6b75a4fa10cae44', 'request_id': 'req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:42.230792 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9c3c148a4cc4f93a6b75a4fa10cae44', 'request_id': 'req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:42.262218 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9c3c148a4cc4f93a6b75a4fa10cae44', 'request_id': 'req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:42.283922 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551971d-1b3d-4487-8769-048fe274077b None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:42.285031 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551971d-1b3d-4487-8769-048fe274077b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:42.285503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551971d-1b3d-4487-8769-048fe274077b None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:42.287623 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 848/1695] 10.222.0.25 () {54 vars in 921 bytes} [Tue Aug 30 14:44:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:44:42.295759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:42.318896 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb admin admin] The token cache is being invalidate because project c9c3c148a4cc4f93a6b75a4fa10cae44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:42.319741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:42.322712 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:42.328285 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 848/1696] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:41 2022] DELETE /identity/v3/projects/c9c3c148a4cc4f93a6b75a4fa10cae44 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:42.333216 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e3c5b-e27e-495d-b6ec-69ebc6120fa1 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:42.333515 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e3c5b-e27e-495d-b6ec-69ebc6120fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:42.333792 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e3c5b-e27e-495d-b6ec-69ebc6120fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:42.385218 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:42.386816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:42.413114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-7d7e3c5b-e27e-495d-b6ec-69ebc6120fa1 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:42.414140 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7d7e3c5b-e27e-495d-b6ec-69ebc6120fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:42.559358 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a4622-da97-41be-8f8f-ef5c8a0ea3fb admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:42.566584 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc10f7f-f42c-4364-af2d-67c5dcb9d3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:42.568820 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc10f7f-f42c-4364-af2d-67c5dcb9d3f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:42.569483 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc10f7f-f42c-4364-af2d-67c5dcb9d3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:42.569903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc10f7f-f42c-4364-af2d-67c5dcb9d3f1 admin admin] PATH_INFO: `/v3/projects/bc68c4d868d645508a5ab248bc58481a` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:42.572056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc10f7f-f42c-4364-af2d-67c5dcb9d3f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc68c4d868d645508a5ab248bc58481a)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:42.574778 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc10f7f-f42c-4364-af2d-67c5dcb9d3f1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:42.678459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 849/1697] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:42.680468 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e02ef-43b9-4cb3-a26b-22f3255a6a8e None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:42.685940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e02ef-43b9-4cb3-a26b-22f3255a6a8e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:42.686600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e02ef-43b9-4cb3-a26b-22f3255a6a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:42.711350 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5cc10f7f-f42c-4364-af2d-67c5dcb9d3f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc68c4d868d645508a5ab248bc58481a', 'request_id': 'req-5cc10f7f-f42c-4364-af2d-67c5dcb9d3f1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:42.722869 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5cc10f7f-f42c-4364-af2d-67c5dcb9d3f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc68c4d868d645508a5ab248bc58481a', 'request_id': 'req-5cc10f7f-f42c-4364-af2d-67c5dcb9d3f1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:42.725092 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-4f1e02ef-43b9-4cb3-a26b-22f3255a6a8e None None] MFA Rules not processed for user `cc95bbdafb0c4f34ad0662bc2e3ba4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:42.727217 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4f1e02ef-43b9-4cb3-a26b-22f3255a6a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:42.733825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5cc10f7f-f42c-4364-af2d-67c5dcb9d3f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc68c4d868d645508a5ab248bc58481a', 'request_id': 'req-5cc10f7f-f42c-4364-af2d-67c5dcb9d3f1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:42.735563 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:42.753385 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5cc10f7f-f42c-4364-af2d-67c5dcb9d3f1 admin admin] The token cache is being invalidate because project bc68c4d868d645508a5ab248bc58481a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:42.753752 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5cc10f7f-f42c-4364-af2d-67c5dcb9d3f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:42.754376 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5cc10f7f-f42c-4364-af2d-67c5dcb9d3f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:42.756737 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 849/1698] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:42 2022] DELETE /identity/v3/projects/bc68c4d868d645508a5ab248bc58481a => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:44:42.758136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5819f-ea0a-44fb-bb3b-1723760b4d85 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:42.758419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5819f-ea0a-44fb-bb3b-1723760b4d85 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:42.758781 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5819f-ea0a-44fb-bb3b-1723760b4d85 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:42.760713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 850/1699] 10.222.0.25 () {54 vars in 921 bytes} [Tue Aug 30 14:44:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:44:42.761860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777c29d-9fc2-43c8-b900-d5cc320b5f8c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:42.762318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777c29d-9fc2-43c8-b900-d5cc320b5f8c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:42.763317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777c29d-9fc2-43c8-b900-d5cc320b5f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:42.784909 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2777c29d-9fc2-43c8-b900-d5cc320b5f8c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:42.786205 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2777c29d-9fc2-43c8-b900-d5cc320b5f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:42.891880 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 850/1700] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Aug 30 14:44:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:42.892719 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e02ef-43b9-4cb3-a26b-22f3255a6a8e None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:42.894820 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4f1e02ef-43b9-4cb3-a26b-22f3255a6a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:42.916568 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 851/1701] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:42.917967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152606c2-14a7-46a2-9728-5e105807159e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:42.918255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152606c2-14a7-46a2-9728-5e105807159e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:42.918524 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152606c2-14a7-46a2-9728-5e105807159e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:42.961918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-152606c2-14a7-46a2-9728-5e105807159e None None] MFA Rules not processed for user `cc95bbdafb0c4f34ad0662bc2e3ba4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:42.963884 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-152606c2-14a7-46a2-9728-5e105807159e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:42.979521 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 852/1702] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Aug 30 14:44:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:42.981865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152606c2-14a7-46a2-9728-5e105807159e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:42.998044 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152606c2-14a7-46a2-9728-5e105807159e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:43.006228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a514b-9f17-4304-b3f4-a5fe3d570c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:43.007364 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a514b-9f17-4304-b3f4-a5fe3d570c1d admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:43.007863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a514b-9f17-4304-b3f4-a5fe3d570c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:43.008303 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a514b-9f17-4304-b3f4-a5fe3d570c1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:43.009300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a514b-9f17-4304-b3f4-a5fe3d570c1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:44:43.009760 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a514b-9f17-4304-b3f4-a5fe3d570c1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:43.013354 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a514b-9f17-4304-b3f4-a5fe3d570c1d admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:43.019932 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e02ef-43b9-4cb3-a26b-22f3255a6a8e None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:43.026213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289fa6a-84bb-4446-88c1-65d081292819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:43.027180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289fa6a-84bb-4446-88c1-65d081292819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:43.027615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289fa6a-84bb-4446-88c1-65d081292819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:43.028078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289fa6a-84bb-4446-88c1-65d081292819 admin admin] PATH_INFO: `/v3/users/e840178a297a4ea28aed82a9ff9f40ad` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:43.029902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8289fa6a-84bb-4446-88c1-65d081292819 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e840178a297a4ea28aed82a9ff9f40ad)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:43.032326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8289fa6a-84bb-4446-88c1-65d081292819 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:43.034155 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 853/1703] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:44:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:44:43.035474 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a514b-9f17-4304-b3f4-a5fe3d570c1d admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:43.038091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c64a514b-9f17-4304-b3f4-a5fe3d570c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:43.062800 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a514b-9f17-4304-b3f4-a5fe3d570c1d admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:43.071468 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b1deb-608c-410f-9e4d-a8cef2d48460 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:43.072155 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b1deb-608c-410f-9e4d-a8cef2d48460 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:43.072423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b1deb-608c-410f-9e4d-a8cef2d48460 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:43.072979 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b1deb-608c-410f-9e4d-a8cef2d48460 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:43.080137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:43.084564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b1deb-608c-410f-9e4d-a8cef2d48460 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:43.086239 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b1deb-608c-410f-9e4d-a8cef2d48460 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:43.108632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8289fa6a-84bb-4446-88c1-65d081292819 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e840178a297a4ea28aed82a9ff9f40ad'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:43.109629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8289fa6a-84bb-4446-88c1-65d081292819 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e840178a297a4ea28aed82a9ff9f40ad'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:43.110423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8289fa6a-84bb-4446-88c1-65d081292819 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e840178a297a4ea28aed82a9ff9f40ad'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:43.113980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 854/1704] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:43.115347 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b1deb-608c-410f-9e4d-a8cef2d48460 service nova] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:43.123788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8289fa6a-84bb-4446-88c1-65d081292819 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e840178a297a4ea28aed82a9ff9f40ad'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:43.129522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b1deb-608c-410f-9e4d-a8cef2d48460 service nova] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:43.135118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553b0f5-f206-46cf-bb3c-f205de3ef6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:43.136264 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553b0f5-f206-46cf-bb3c-f205de3ef6c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:43.136264 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553b0f5-f206-46cf-bb3c-f205de3ef6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:43.136673 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553b0f5-f206-46cf-bb3c-f205de3ef6c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:43.138525 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553b0f5-f206-46cf-bb3c-f205de3ef6c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:43.140926 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553b0f5-f206-46cf-bb3c-f205de3ef6c8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:43.148263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8289fa6a-84bb-4446-88c1-65d081292819 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e840178a297a4ea28aed82a9ff9f40ad'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:43.169362 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 851/1705] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:42 2022] DELETE /identity/v3/users/e840178a297a4ea28aed82a9ff9f40ad => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:43.184639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 855/1706] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:43 2022] POST /identity/v3/projects => generated 369 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:43.185880 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289fa6a-84bb-4446-88c1-65d081292819 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:43.187874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-8289fa6a-84bb-4446-88c1-65d081292819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:43.189954 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553b0f5-f206-46cf-bb3c-f205de3ef6c8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:43.192517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5553b0f5-f206-46cf-bb3c-f205de3ef6c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:43.229467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:43.281408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553b0f5-f206-46cf-bb3c-f205de3ef6c8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:43.287109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f169b-843e-44f6-92f1-c04b7fa8f16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:43.287821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f169b-843e-44f6-92f1-c04b7fa8f16a admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:43.288192 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f169b-843e-44f6-92f1-c04b7fa8f16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:43.288538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f169b-843e-44f6-92f1-c04b7fa8f16a admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:43.289298 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f169b-843e-44f6-92f1-c04b7fa8f16a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:43.291338 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f169b-843e-44f6-92f1-c04b7fa8f16a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:43.324103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 856/1707] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:44:43 2022] POST /identity/v3/users => generated 285 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:44:43.325292 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f169b-843e-44f6-92f1-c04b7fa8f16a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:43.327550 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-821f169b-843e-44f6-92f1-c04b7fa8f16a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:43.341725 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f169b-843e-44f6-92f1-c04b7fa8f16a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:43.347913 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289fa6a-84bb-4446-88c1-65d081292819 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:43.353827 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29272e82-9c4e-4d7c-8e58-d4d518a51a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:43.354628 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29272e82-9c4e-4d7c-8e58-d4d518a51a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:43.354973 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29272e82-9c4e-4d7c-8e58-d4d518a51a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:43.355305 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29272e82-9c4e-4d7c-8e58-d4d518a51a24 admin admin] PATH_INFO: `/v3/projects/561fa2cd4eb340358851628d57a98118` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:43.357962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29272e82-9c4e-4d7c-8e58-d4d518a51a24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=561fa2cd4eb340358851628d57a98118)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:43.359355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29272e82-9c4e-4d7c-8e58-d4d518a51a24 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:43.445750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-29272e82-9c4e-4d7c-8e58-d4d518a51a24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '561fa2cd4eb340358851628d57a98118', 'request_id': 'req-29272e82-9c4e-4d7c-8e58-d4d518a51a24'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:43.469101 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-29272e82-9c4e-4d7c-8e58-d4d518a51a24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '561fa2cd4eb340358851628d57a98118', 'request_id': 'req-29272e82-9c4e-4d7c-8e58-d4d518a51a24'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:43.488861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-29272e82-9c4e-4d7c-8e58-d4d518a51a24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '561fa2cd4eb340358851628d57a98118', 'request_id': 'req-29272e82-9c4e-4d7c-8e58-d4d518a51a24'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:43.500774 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a914a3-87a9-454d-8dbf-2e91b8492461 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:43.501512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a914a3-87a9-454d-8dbf-2e91b8492461 service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:43.501956 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a914a3-87a9-454d-8dbf-2e91b8492461 service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:43.513113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-29272e82-9c4e-4d7c-8e58-d4d518a51a24 admin admin] The token cache is being invalidate because project 561fa2cd4eb340358851628d57a98118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:43.513592 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-29272e82-9c4e-4d7c-8e58-d4d518a51a24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:43.514098 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a914a3-87a9-454d-8dbf-2e91b8492461 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:43.515385 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-29272e82-9c4e-4d7c-8e58-d4d518a51a24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:43.517951 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 857/1708] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:43 2022] DELETE /identity/v3/projects/561fa2cd4eb340358851628d57a98118 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:43.520606 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29272e82-9c4e-4d7c-8e58-d4d518a51a24 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:43.524354 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a5a914a3-87a9-454d-8dbf-2e91b8492461 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:43.525538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-29272e82-9c4e-4d7c-8e58-d4d518a51a24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:43.539968 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a914a3-87a9-454d-8dbf-2e91b8492461 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:43.541971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a914a3-87a9-454d-8dbf-2e91b8492461 service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:43.568882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 852/1709] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:43.574084 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6f0a3-115e-4f3f-94ec-3eea3b7d253e None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:43.578143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6f0a3-115e-4f3f-94ec-3eea3b7d253e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:43.581679 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6f0a3-115e-4f3f-94ec-3eea3b7d253e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:43.635708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-35c6f0a3-115e-4f3f-94ec-3eea3b7d253e None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:43.637890 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-35c6f0a3-115e-4f3f-94ec-3eea3b7d253e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:43.646873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:43.673548 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29272e82-9c4e-4d7c-8e58-d4d518a51a24 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:43.679806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13af1e-650f-4cd4-9b8f-dd359fc71dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:43.681402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13af1e-650f-4cd4-9b8f-dd359fc71dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:43.681684 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13af1e-650f-4cd4-9b8f-dd359fc71dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:43.681958 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13af1e-650f-4cd4-9b8f-dd359fc71dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:43.682864 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13af1e-650f-4cd4-9b8f-dd359fc71dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:43.685556 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13af1e-650f-4cd4-9b8f-dd359fc71dd4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:43.706246 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 858/1710] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:43.707807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13af1e-650f-4cd4-9b8f-dd359fc71dd4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:43.733832 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13af1e-650f-4cd4-9b8f-dd359fc71dd4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:43.747170 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04288a-c3d7-4296-9489-fa715b79d8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:43.748242 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04288a-c3d7-4296-9489-fa715b79d8ef admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:43.748709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04288a-c3d7-4296-9489-fa715b79d8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:43.749161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04288a-c3d7-4296-9489-fa715b79d8ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:43.750313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04288a-c3d7-4296-9489-fa715b79d8ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:43.754437 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04288a-c3d7-4296-9489-fa715b79d8ef admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:43.779435 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 859/1711] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:43.784645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04288a-c3d7-4296-9489-fa715b79d8ef admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:43.805822 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04288a-c3d7-4296-9489-fa715b79d8ef admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:43.806107 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 853/1712] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:43.813774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b47e1-b7c9-44b3-998b-3952211631bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:43.814927 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b47e1-b7c9-44b3-998b-3952211631bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:43.815409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b47e1-b7c9-44b3-998b-3952211631bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:43.815873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b47e1-b7c9-44b3-998b-3952211631bb admin admin] PATH_INFO: `/v3/projects/07d9f3b730824b389fbd3707b805e25c/users/74166f0a6a754cd6b08d2aab14a3cf4e/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:43.817705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce9497-0d27-4a0c-b37a-2708cb2dff52 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:43.818408 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce9497-0d27-4a0c-b37a-2708cb2dff52 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:43.819048 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce9497-0d27-4a0c-b37a-2708cb2dff52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:43.847895 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b47e1-b7c9-44b3-998b-3952211631bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=07d9f3b730824b389fbd3707b805e25c, user_id=74166f0a6a754cd6b08d2aab14a3cf4e, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:43.853649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b47e1-b7c9-44b3-998b-3952211631bb admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:43.856117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-02ce9497-0d27-4a0c-b37a-2708cb2dff52 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:43.859835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-02ce9497-0d27-4a0c-b37a-2708cb2dff52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:43.902522 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 854/1714] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:43.915447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 860/1714] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:43 2022] PUT /identity/v3/projects/07d9f3b730824b389fbd3707b805e25c/users/74166f0a6a754cd6b08d2aab14a3cf4e/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:43.919116 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce9497-0d27-4a0c-b37a-2708cb2dff52 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:43.927410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6b9ee-0339-473c-bf59-0ea0f4fc6b61 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:43.927410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6b9ee-0339-473c-bf59-0ea0f4fc6b61 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:43.927410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6b9ee-0339-473c-bf59-0ea0f4fc6b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:43.952544 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce9497-0d27-4a0c-b37a-2708cb2dff52 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:43.972877 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-78e6b9ee-0339-473c-bf59-0ea0f4fc6b61 None None] MFA Rules not processed for user `74166f0a6a754cd6b08d2aab14a3cf4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:43.974425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-78e6b9ee-0339-473c-bf59-0ea0f4fc6b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:44.038550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:44.072421 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9699db-5917-41da-acee-6b9753f251a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:44.073364 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9699db-5917-41da-acee-6b9753f251a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:44.073652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9699db-5917-41da-acee-6b9753f251a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:44.073951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9699db-5917-41da-acee-6b9753f251a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:44.074808 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9699db-5917-41da-acee-6b9753f251a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:44:44.075116 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9699db-5917-41da-acee-6b9753f251a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:44.076921 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9699db-5917-41da-acee-6b9753f251a5 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:44.086349 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 861/1715] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:44:44.095737 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6b9ee-0339-473c-bf59-0ea0f4fc6b61 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:44.102761 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 855/1716] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:44:44.107786 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9699db-5917-41da-acee-6b9753f251a5 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:44.112316 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6b9ee-0339-473c-bf59-0ea0f4fc6b61 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:44.118661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122019e7-b448-4cea-8666-9dc15c5fdbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:44.119371 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122019e7-b448-4cea-8666-9dc15c5fdbb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:44.119720 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122019e7-b448-4cea-8666-9dc15c5fdbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:44.120143 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122019e7-b448-4cea-8666-9dc15c5fdbb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:44.120810 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122019e7-b448-4cea-8666-9dc15c5fdbb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:44.122841 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122019e7-b448-4cea-8666-9dc15c5fdbb6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:44.138555 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9699db-5917-41da-acee-6b9753f251a5 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:44.149096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2b815-94ca-4bb0-9925-c85025a31fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:44.150165 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2b815-94ca-4bb0-9925-c85025a31fdc admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:44.150656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2b815-94ca-4bb0-9925-c85025a31fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:44.151200 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2b815-94ca-4bb0-9925-c85025a31fdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:44.152406 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2b815-94ca-4bb0-9925-c85025a31fdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:44.157762 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2b815-94ca-4bb0-9925-c85025a31fdc admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:44.178074 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 862/1717] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:44 2022] POST /identity/v3/projects => generated 367 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:44:44.179068 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122019e7-b448-4cea-8666-9dc15c5fdbb6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:44.181337 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-122019e7-b448-4cea-8666-9dc15c5fdbb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:44.224083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 856/1718] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:44 2022] POST /identity/v3/projects => generated 363 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:44:44.224083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2b815-94ca-4bb0-9925-c85025a31fdc admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:44.279556 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122019e7-b448-4cea-8666-9dc15c5fdbb6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:44.303080 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:44.356806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2b815-94ca-4bb0-9925-c85025a31fdc admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:44.366875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5972d6-aa19-47bf-8b25-fd9ea1139416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:44.367974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5972d6-aa19-47bf-8b25-fd9ea1139416 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:44.368497 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5972d6-aa19-47bf-8b25-fd9ea1139416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:44.369570 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5972d6-aa19-47bf-8b25-fd9ea1139416 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:44.370585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5972d6-aa19-47bf-8b25-fd9ea1139416 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:44.374870 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5972d6-aa19-47bf-8b25-fd9ea1139416 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:44.377392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:44.378434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:44.378888 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:44.379522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627 admin admin] PATH_INFO: `/v3/projects/8a813b391ab84dd686adb175da862e95` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:44.381801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a813b391ab84dd686adb175da862e95)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:44.384849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:44.422956 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 857/1719] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:44:44 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:44.430676 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5972d6-aa19-47bf-8b25-fd9ea1139416 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:44.453773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5972d6-aa19-47bf-8b25-fd9ea1139416 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:44.481630 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a813b391ab84dd686adb175da862e95', 'request_id': 'req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:44.494781 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a813b391ab84dd686adb175da862e95', 'request_id': 'req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:44.509570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a813b391ab84dd686adb175da862e95', 'request_id': 'req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:44.529058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627 admin admin] The token cache is being invalidate because project 8a813b391ab84dd686adb175da862e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:44.529540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:44.530644 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:44.532754 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 863/1720] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:44 2022] DELETE /identity/v3/projects/8a813b391ab84dd686adb175da862e95 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:44.535084 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:44.539370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:44.556380 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43acdb-2289-4766-8862-d8c6b31b8ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:44.557404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43acdb-2289-4766-8862-d8c6b31b8ecc admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:44.557854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43acdb-2289-4766-8862-d8c6b31b8ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:44.558279 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43acdb-2289-4766-8862-d8c6b31b8ecc admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:44.559374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43acdb-2289-4766-8862-d8c6b31b8ecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:44.563570 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43acdb-2289-4766-8862-d8c6b31b8ecc admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:44.601802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 858/1721] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:44:44 2022] POST /identity/v3/users => generated 282 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:44.603356 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04595236-561b-4d20-8f4e-bdc01b37d113 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:44.603803 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04595236-561b-4d20-8f4e-bdc01b37d113 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:44.604242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04595236-561b-4d20-8f4e-bdc01b37d113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:44.627189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-04595236-561b-4d20-8f4e-bdc01b37d113 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:44.628536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-04595236-561b-4d20-8f4e-bdc01b37d113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:44.639330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 859/1722] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:44.640269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04595236-561b-4d20-8f4e-bdc01b37d113 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:44.642562 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-04595236-561b-4d20-8f4e-bdc01b37d113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:44.659017 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04595236-561b-4d20-8f4e-bdc01b37d113 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:44.661851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5d6f0-0f8f-4425-b0fe-8c95fa6d1627 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:44.664920 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c3e68-6e47-4056-9cb7-93177c980fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:44.665764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c3e68-6e47-4056-9cb7-93177c980fba admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:44.666053 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c3e68-6e47-4056-9cb7-93177c980fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:44.666565 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c3e68-6e47-4056-9cb7-93177c980fba admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:44.667039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c3e68-6e47-4056-9cb7-93177c980fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:44.668778 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c3e68-6e47-4056-9cb7-93177c980fba admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:44.669966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca856a9a-7128-446b-9096-247cf2456997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:44.670906 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca856a9a-7128-446b-9096-247cf2456997 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:44.671283 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca856a9a-7128-446b-9096-247cf2456997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:44.671760 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca856a9a-7128-446b-9096-247cf2456997 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:44.672932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca856a9a-7128-446b-9096-247cf2456997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:44.675073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca856a9a-7128-446b-9096-247cf2456997 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:44.686538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 860/1723] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:44.688907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fe7b0-3823-4148-ae09-c77efb085be8 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:44.689403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fe7b0-3823-4148-ae09-c77efb085be8 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:44.689938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fe7b0-3823-4148-ae09-c77efb085be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:44.694682 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 864/1724] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:44.696505 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca856a9a-7128-446b-9096-247cf2456997 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:44.712242 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca856a9a-7128-446b-9096-247cf2456997 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:44.726787 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fb538-9771-4ce8-ab14-a71a44008d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:44.728023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fb538-9771-4ce8-ab14-a71a44008d32 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:44.728616 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fb538-9771-4ce8-ab14-a71a44008d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:44.730752 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fb538-9771-4ce8-ab14-a71a44008d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:44.734387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fb538-9771-4ce8-ab14-a71a44008d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:44.741134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fb538-9771-4ce8-ab14-a71a44008d32 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:44.764250 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 865/1725] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:44.765366 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-736fe7b0-3823-4148-ae09-c77efb085be8 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:44.766228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fb538-9771-4ce8-ab14-a71a44008d32 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:44.767188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-736fe7b0-3823-4148-ae09-c77efb085be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:44.784027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 861/1726] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:44.789565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fb538-9771-4ce8-ab14-a71a44008d32 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:44.797593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57955c-5aa6-47e7-a395-c9eb788e45eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:44.797907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fe7b0-3823-4148-ae09-c77efb085be8 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:44.799212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57955c-5aa6-47e7-a395-c9eb788e45eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:44.799838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57955c-5aa6-47e7-a395-c9eb788e45eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:44.800477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57955c-5aa6-47e7-a395-c9eb788e45eb admin admin] PATH_INFO: `/v3/projects/06b30445a1af40239a9360cd6c8bebb6/users/de28c6bf20b9400d8970e06c963e04a6/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:44.836028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57955c-5aa6-47e7-a395-c9eb788e45eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=06b30445a1af40239a9360cd6c8bebb6, user_id=de28c6bf20b9400d8970e06c963e04a6, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:44.840297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fe7b0-3823-4148-ae09-c77efb085be8 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:44.841171 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57955c-5aa6-47e7-a395-c9eb788e45eb admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:44.850987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86845a4e-ee17-41f0-a805-5583b4dfd929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:44.851935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86845a4e-ee17-41f0-a805-5583b4dfd929 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:44.852606 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86845a4e-ee17-41f0-a805-5583b4dfd929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:44.853071 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86845a4e-ee17-41f0-a805-5583b4dfd929 admin admin] PATH_INFO: `/v3/projects/90aefd1c4ec24087a93eb07b4a62c94e/users/14fe67a054de49a9af8e240ef5187db5/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:44.857610 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:44.876150 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 866/1727] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:44 2022] PUT /identity/v3/projects/06b30445a1af40239a9360cd6c8bebb6/users/de28c6bf20b9400d8970e06c963e04a6/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:44.877902 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57955c-5aa6-47e7-a395-c9eb788e45eb admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:44.887432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86845a4e-ee17-41f0-a805-5583b4dfd929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90aefd1c4ec24087a93eb07b4a62c94e, user_id=14fe67a054de49a9af8e240ef5187db5, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:44.904400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86845a4e-ee17-41f0-a805-5583b4dfd929 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:44.923289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:44.962175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 862/1728] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:44 2022] PUT /identity/v3/projects/90aefd1c4ec24087a93eb07b4a62c94e/users/14fe67a054de49a9af8e240ef5187db5/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:44.965703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86845a4e-ee17-41f0-a805-5583b4dfd929 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:45.083278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57955c-5aa6-47e7-a395-c9eb788e45eb admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:45.128208 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86845a4e-ee17-41f0-a805-5583b4dfd929 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:45.140610 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1401a-67a9-410c-8309-f28aecebdca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:45.141748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1401a-67a9-410c-8309-f28aecebdca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:45.142247 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1401a-67a9-410c-8309-f28aecebdca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:45.142740 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1401a-67a9-410c-8309-f28aecebdca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:45.143776 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1401a-67a9-410c-8309-f28aecebdca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:45.145933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1401a-67a9-410c-8309-f28aecebdca6 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:45.182702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 863/1729] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:45.182702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fc82e-83ce-4bf4-bc85-90cc615b67dd None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:45.182702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fc82e-83ce-4bf4-bc85-90cc615b67dd None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:45.182702 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fc82e-83ce-4bf4-bc85-90cc615b67dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:45.208109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b91719-5ea1-48c7-adf6-1519ae80a5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:45.209488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b91719-5ea1-48c7-adf6-1519ae80a5e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:45.210186 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b91719-5ea1-48c7-adf6-1519ae80a5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:45.210856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b91719-5ea1-48c7-adf6-1519ae80a5e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:45.212345 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b91719-5ea1-48c7-adf6-1519ae80a5e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:44:45.213034 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b91719-5ea1-48c7-adf6-1519ae80a5e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:45.216348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b91719-5ea1-48c7-adf6-1519ae80a5e8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:45.249873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 867/1730] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:44:45.249873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b91719-5ea1-48c7-adf6-1519ae80a5e8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:45.266736 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b91719-5ea1-48c7-adf6-1519ae80a5e8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:45.272492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-758fc82e-83ce-4bf4-bc85-90cc615b67dd None None] MFA Rules not processed for user `14fe67a054de49a9af8e240ef5187db5`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:45.274860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5ff47-9686-4b9a-a5e0-81be727aeaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:45.275032 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-758fc82e-83ce-4bf4-bc85-90cc615b67dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:45.276071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5ff47-9686-4b9a-a5e0-81be727aeaa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:45.276552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5ff47-9686-4b9a-a5e0-81be727aeaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:45.277019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5ff47-9686-4b9a-a5e0-81be727aeaa4 admin admin] PATH_INFO: `/v3/projects/06b30445a1af40239a9360cd6c8bebb6/users/de28c6bf20b9400d8970e06c963e04a6/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:45.280569 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5ff47-9686-4b9a-a5e0-81be727aeaa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06b30445a1af40239a9360cd6c8bebb6, user_id=de28c6bf20b9400d8970e06c963e04a6, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:45.288038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5ff47-9686-4b9a-a5e0-81be727aeaa4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:45.308836 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 868/1731] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:45 2022] PUT /identity/v3/projects/06b30445a1af40239a9360cd6c8bebb6/users/de28c6bf20b9400d8970e06c963e04a6/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:45.311490 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5ff47-9686-4b9a-a5e0-81be727aeaa4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:45.443889 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 864/1732] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:45.446438 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d4f85-663f-44cb-ac21-69edf498521d None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:45.446837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d4f85-663f-44cb-ac21-69edf498521d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:45.447092 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d4f85-663f-44cb-ac21-69edf498521d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:45.490355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5ff47-9686-4b9a-a5e0-81be727aeaa4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:45.505094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51490f-6c7f-4f28-96f5-08fd8b237e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:45.505753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51490f-6c7f-4f28-96f5-08fd8b237e38 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:45.506030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51490f-6c7f-4f28-96f5-08fd8b237e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:45.506304 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51490f-6c7f-4f28-96f5-08fd8b237e38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:45.507282 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51490f-6c7f-4f28-96f5-08fd8b237e38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:45.509464 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51490f-6c7f-4f28-96f5-08fd8b237e38 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:45.537053 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a86d4f85-663f-44cb-ac21-69edf498521d None None] MFA Rules not processed for user `de28c6bf20b9400d8970e06c963e04a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:45.538734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a86d4f85-663f-44cb-ac21-69edf498521d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:45.557430 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 869/1733] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:45 2022] POST /identity/v3/projects => generated 379 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:45.558648 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51490f-6c7f-4f28-96f5-08fd8b237e38 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:45.615771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:45.725967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51490f-6c7f-4f28-96f5-08fd8b237e38 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:45.735896 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:45.741090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be137b87-f1e4-454d-883f-e23861c8ed07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:45.741827 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be137b87-f1e4-454d-883f-e23861c8ed07 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:45.742100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be137b87-f1e4-454d-883f-e23861c8ed07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:45.742365 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be137b87-f1e4-454d-883f-e23861c8ed07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:45.749634 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be137b87-f1e4-454d-883f-e23861c8ed07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:45.755429 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be137b87-f1e4-454d-883f-e23861c8ed07 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:45.767464 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 865/1734] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:45 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:45.768443 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d4f85-663f-44cb-ac21-69edf498521d None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:45.792796 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d4f85-663f-44cb-ac21-69edf498521d None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:45.803199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f4934-b194-43bc-87f6-3bae66115d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:45.803944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f4934-b194-43bc-87f6-3bae66115d67 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:45.804464 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f4934-b194-43bc-87f6-3bae66115d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:45.804784 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f4934-b194-43bc-87f6-3bae66115d67 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:45.806458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f4934-b194-43bc-87f6-3bae66115d67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:45.818876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f4934-b194-43bc-87f6-3bae66115d67 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:45.830878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 870/1735] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:45 2022] POST /identity/v3/projects => generated 365 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:45.831934 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be137b87-f1e4-454d-883f-e23861c8ed07 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:45.841082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-be137b87-f1e4-454d-883f-e23861c8ed07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:45.897132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 866/1736] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:44:45 2022] POST /identity/v3/users => generated 290 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:45.897132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f4934-b194-43bc-87f6-3bae66115d67 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:45.909981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f4934-b194-43bc-87f6-3bae66115d67 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:46.066965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5837c-4c04-4645-a28d-32b61874d0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:46.067694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be137b87-f1e4-454d-883f-e23861c8ed07 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:46.068430 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5837c-4c04-4645-a28d-32b61874d0b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:46.072404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5837c-4c04-4645-a28d-32b61874d0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:46.076527 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5837c-4c04-4645-a28d-32b61874d0b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:46.081328 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d9e2a-c624-4aa3-8ad8-254f2d9deacf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:46.084712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d5837c-4c04-4645-a28d-32b61874d0b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:46.096737 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d9e2a-c624-4aa3-8ad8-254f2d9deacf service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:46.097434 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d9e2a-c624-4aa3-8ad8-254f2d9deacf service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:46.097774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d9e2a-c624-4aa3-8ad8-254f2d9deacf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:46.100866 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d5837c-4c04-4645-a28d-32b61874d0b1 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:46.250883 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 867/1737] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:44:45 2022] POST /identity/v3/users => generated 289 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:46.252966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5837c-4c04-4645-a28d-32b61874d0b1 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:46.332893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d9e2a-c624-4aa3-8ad8-254f2d9deacf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:46.335201 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d9e2a-c624-4aa3-8ad8-254f2d9deacf service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:46.338826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5837c-4c04-4645-a28d-32b61874d0b1 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:46.348813 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 871/1738] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:46.350009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d9e2a-c624-4aa3-8ad8-254f2d9deacf service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:46.361466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395310c-f1eb-4528-9115-c88b02fea9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:46.363672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395310c-f1eb-4528-9115-c88b02fea9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:46.364156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395310c-f1eb-4528-9115-c88b02fea9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:46.364605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395310c-f1eb-4528-9115-c88b02fea9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:46.369200 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d9e2a-c624-4aa3-8ad8-254f2d9deacf service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:46.370430 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395310c-f1eb-4528-9115-c88b02fea9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:46.377583 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395310c-f1eb-4528-9115-c88b02fea9d7 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:46.379062 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73a343-16d8-4d8d-a1b3-7385659b01d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:46.379981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73a343-16d8-4d8d-a1b3-7385659b01d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:46.380550 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73a343-16d8-4d8d-a1b3-7385659b01d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:46.381090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73a343-16d8-4d8d-a1b3-7385659b01d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:46.382236 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73a343-16d8-4d8d-a1b3-7385659b01d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:46.385305 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73a343-16d8-4d8d-a1b3-7385659b01d4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:46.401438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 872/1739] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:46.405646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73a343-16d8-4d8d-a1b3-7385659b01d4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:46.419564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73a343-16d8-4d8d-a1b3-7385659b01d4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:46.426193 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8b9af-b7b6-4699-88c2-b4eec5c7082f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:46.427325 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8b9af-b7b6-4699-88c2-b4eec5c7082f admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:46.427833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8b9af-b7b6-4699-88c2-b4eec5c7082f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:46.428439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8b9af-b7b6-4699-88c2-b4eec5c7082f admin admin] PATH_INFO: `/v3/projects/4c95826184d84a2f9e01cbc43c0e7ee9/users/dd570edc12614c20acaba510ecc9a61e/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:46.468521 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8b9af-b7b6-4699-88c2-b4eec5c7082f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c95826184d84a2f9e01cbc43c0e7ee9, user_id=dd570edc12614c20acaba510ecc9a61e, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:46.471505 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8b9af-b7b6-4699-88c2-b4eec5c7082f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:46.496341 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 868/1740] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:46.510102 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 873/1741] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:46 2022] PUT /identity/v3/projects/4c95826184d84a2f9e01cbc43c0e7ee9/users/dd570edc12614c20acaba510ecc9a61e/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:44:46.510102 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8b9af-b7b6-4699-88c2-b4eec5c7082f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:46.516764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395310c-f1eb-4528-9115-c88b02fea9d7 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:46.567162 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395310c-f1eb-4528-9115-c88b02fea9d7 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:46.581258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:46.651556 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8b9af-b7b6-4699-88c2-b4eec5c7082f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:46.657088 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a1545-d752-4d4c-b825-9dd5b54b9505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:46.657930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a1545-d752-4d4c-b825-9dd5b54b9505 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:46.658206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a1545-d752-4d4c-b825-9dd5b54b9505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:46.658537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a1545-d752-4d4c-b825-9dd5b54b9505 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:46.659160 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a1545-d752-4d4c-b825-9dd5b54b9505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:46.661625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a1545-d752-4d4c-b825-9dd5b54b9505 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:46.679621 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 874/1742] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:46.680943 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a1545-d752-4d4c-b825-9dd5b54b9505 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:46.695418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a1545-d752-4d4c-b825-9dd5b54b9505 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:46.702598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e828f27-6be5-4ca0-8bf2-7d8d7c5a0a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:46.703809 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e828f27-6be5-4ca0-8bf2-7d8d7c5a0a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:46.704371 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e828f27-6be5-4ca0-8bf2-7d8d7c5a0a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:46.704905 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e828f27-6be5-4ca0-8bf2-7d8d7c5a0a70 admin admin] PATH_INFO: `/v3/projects/93ca9c31302141c3bcece7cc90855c26/users/b9dc342fbc6a47b8a02c8172c77f7974/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:46.730421 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e828f27-6be5-4ca0-8bf2-7d8d7c5a0a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ca9c31302141c3bcece7cc90855c26, user_id=b9dc342fbc6a47b8a02c8172c77f7974, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:46.735219 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e828f27-6be5-4ca0-8bf2-7d8d7c5a0a70 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:46.784197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 875/1743] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:46 2022] PUT /identity/v3/projects/93ca9c31302141c3bcece7cc90855c26/users/b9dc342fbc6a47b8a02c8172c77f7974/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:46.818464 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87336e-f3bf-4464-bc48-dfdc7b159d98 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:46.818753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87336e-f3bf-4464-bc48-dfdc7b159d98 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:46.819216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87336e-f3bf-4464-bc48-dfdc7b159d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:46.882537 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b33ba6-0bd6-43f2-b250-994ad190c414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:46.885338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b33ba6-0bd6-43f2-b250-994ad190c414 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:46.886210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b33ba6-0bd6-43f2-b250-994ad190c414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:46.887341 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b33ba6-0bd6-43f2-b250-994ad190c414 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:46.888941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b33ba6-0bd6-43f2-b250-994ad190c414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:46.893382 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b33ba6-0bd6-43f2-b250-994ad190c414 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:46.912090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7f87336e-f3bf-4464-bc48-dfdc7b159d98 None None] MFA Rules not processed for user `b9dc342fbc6a47b8a02c8172c77f7974`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:46.917653 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7f87336e-f3bf-4464-bc48-dfdc7b159d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:46.949413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 869/1744] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:46.953864 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b33ba6-0bd6-43f2-b250-994ad190c414 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:47.010268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b33ba6-0bd6-43f2-b250-994ad190c414 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:47.194248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 876/1745] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:44:47.198917 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87336e-f3bf-4464-bc48-dfdc7b159d98 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:47.285359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:47.334996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282ea1e-a938-456e-9f5e-96431c7df5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:47.338278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282ea1e-a938-456e-9f5e-96431c7df5fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:47.339818 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282ea1e-a938-456e-9f5e-96431c7df5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:47.343942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282ea1e-a938-456e-9f5e-96431c7df5fc admin admin] PATH_INFO: `/v3/projects/4c95826184d84a2f9e01cbc43c0e7ee9/users/dd570edc12614c20acaba510ecc9a61e/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:47.375579 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282ea1e-a938-456e-9f5e-96431c7df5fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c95826184d84a2f9e01cbc43c0e7ee9, user_id=dd570edc12614c20acaba510ecc9a61e, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:47.392867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282ea1e-a938-456e-9f5e-96431c7df5fc admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:47.462331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 870/1746] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:46 2022] PUT /identity/v3/projects/4c95826184d84a2f9e01cbc43c0e7ee9/users/dd570edc12614c20acaba510ecc9a61e/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:47.474963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6551fa-377a-4f7a-870b-d3204549f800 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:47.479650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6551fa-377a-4f7a-870b-d3204549f800 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:47.480286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6551fa-377a-4f7a-870b-d3204549f800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:47.521495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87336e-f3bf-4464-bc48-dfdc7b159d98 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:47.614701 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-df6551fa-377a-4f7a-870b-d3204549f800 None None] MFA Rules not processed for user `dd570edc12614c20acaba510ecc9a61e`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:47.617378 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-df6551fa-377a-4f7a-870b-d3204549f800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:47.653819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:47.946737 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e81766-4273-4141-9645-3de08b273426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:47.953495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e81766-4273-4141-9645-3de08b273426 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:47.956331 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e81766-4273-4141-9645-3de08b273426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:47.963307 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e81766-4273-4141-9645-3de08b273426 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:47.966025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e81766-4273-4141-9645-3de08b273426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:47.982926 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e81766-4273-4141-9645-3de08b273426 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:47.997111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 871/1747] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 523 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:48.043527 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6551fa-377a-4f7a-870b-d3204549f800 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:48.217106 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 877/1748] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:47 2022] POST /identity/v3/projects => generated 379 bytes in 1018 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:48.222357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e81766-4273-4141-9645-3de08b273426 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:48.284800 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6551fa-377a-4f7a-870b-d3204549f800 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:48.485838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748c53e-4290-4362-a38a-c7856a049a8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:48.485838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748c53e-4290-4362-a38a-c7856a049a8e service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:48.486353 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748c53e-4290-4362-a38a-c7856a049a8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:48.486654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748c53e-4290-4362-a38a-c7856a049a8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:48.499007 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e81766-4273-4141-9645-3de08b273426 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:48.510888 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8fa6d-f428-46ef-8086-b63fc728aed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:48.511546 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8fa6d-f428-46ef-8086-b63fc728aed1 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:48.512749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8fa6d-f428-46ef-8086-b63fc728aed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:48.512945 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8fa6d-f428-46ef-8086-b63fc728aed1 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:48.513854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8fa6d-f428-46ef-8086-b63fc728aed1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:48.516954 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8fa6d-f428-46ef-8086-b63fc728aed1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:48.596348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 878/1749] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:44:48 2022] POST /identity/v3/users => generated 296 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:48.603117 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8fa6d-f428-46ef-8086-b63fc728aed1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:48.609478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:48.631422 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8fa6d-f428-46ef-8086-b63fc728aed1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:48.647000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebb48d-3c6b-43cf-a1b7-706d152afd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:48.647741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebb48d-3c6b-43cf-a1b7-706d152afd94 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:48.648051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebb48d-3c6b-43cf-a1b7-706d152afd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:48.648327 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebb48d-3c6b-43cf-a1b7-706d152afd94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:48.649000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ebb48d-3c6b-43cf-a1b7-706d152afd94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:48.653680 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ebb48d-3c6b-43cf-a1b7-706d152afd94 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:48.656625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748c53e-4290-4362-a38a-c7856a049a8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:48.658512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748c53e-4290-4362-a38a-c7856a049a8e service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:48.688615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 879/1750] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:48.693386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 872/1751] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:48.697006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebb48d-3c6b-43cf-a1b7-706d152afd94 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:48.736820 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebb48d-3c6b-43cf-a1b7-706d152afd94 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:48.747273 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b2768-9518-43f6-b2ac-f562bfc38708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:48.748407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b2768-9518-43f6-b2ac-f562bfc38708 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:48.748729 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b2768-9518-43f6-b2ac-f562bfc38708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:48.749037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b2768-9518-43f6-b2ac-f562bfc38708 admin admin] PATH_INFO: `/v3/projects/1b337c4d786e4a5aaec8afe4a6393040/users/8debab7fc32a47839138ef61ef8d8ec9/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:48.817178 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b2768-9518-43f6-b2ac-f562bfc38708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b337c4d786e4a5aaec8afe4a6393040, user_id=8debab7fc32a47839138ef61ef8d8ec9, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:48.844469 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b2768-9518-43f6-b2ac-f562bfc38708 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:48.944383 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748c53e-4290-4362-a38a-c7856a049a8e service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:48.946359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 880/1752] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:48 2022] PUT /identity/v3/projects/1b337c4d786e4a5aaec8afe4a6393040/users/8debab7fc32a47839138ef61ef8d8ec9/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:48.973682 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b2768-9518-43f6-b2ac-f562bfc38708 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:48.976976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748c53e-4290-4362-a38a-c7856a049a8e service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:49.155690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b2768-9518-43f6-b2ac-f562bfc38708 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:49.162381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2549d1-6293-4ce6-805f-a0517c95f2d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:49.162381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2549d1-6293-4ce6-805f-a0517c95f2d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:49.162850 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2549d1-6293-4ce6-805f-a0517c95f2d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:49.163228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2549d1-6293-4ce6-805f-a0517c95f2d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:49.203263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1273e13-b254-4d3e-862c-68d56891caf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:49.204229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1273e13-b254-4d3e-862c-68d56891caf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:49.204686 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1273e13-b254-4d3e-862c-68d56891caf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:49.205148 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1273e13-b254-4d3e-862c-68d56891caf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:49.206011 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1273e13-b254-4d3e-862c-68d56891caf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:49.207670 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:49.208303 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1273e13-b254-4d3e-862c-68d56891caf5 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:49.239103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 873/1753] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:49.241414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1273e13-b254-4d3e-862c-68d56891caf5 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:49.268105 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1273e13-b254-4d3e-862c-68d56891caf5 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:49.283986 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82efa3-20d4-4aee-bfbc-9ec51fc9caca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:49.285023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82efa3-20d4-4aee-bfbc-9ec51fc9caca admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:49.285452 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82efa3-20d4-4aee-bfbc-9ec51fc9caca admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:49.285792 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82efa3-20d4-4aee-bfbc-9ec51fc9caca admin admin] PATH_INFO: `/v3/projects/1b337c4d786e4a5aaec8afe4a6393040/users/8debab7fc32a47839138ef61ef8d8ec9/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:49.296929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82efa3-20d4-4aee-bfbc-9ec51fc9caca admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b337c4d786e4a5aaec8afe4a6393040, user_id=8debab7fc32a47839138ef61ef8d8ec9, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:49.304869 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2549d1-6293-4ce6-805f-a0517c95f2d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:49.305685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82efa3-20d4-4aee-bfbc-9ec51fc9caca admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:49.305773 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2549d1-6293-4ce6-805f-a0517c95f2d1 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:49.338007 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 881/1754] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:49.372517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 874/1755] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:49 2022] PUT /identity/v3/projects/1b337c4d786e4a5aaec8afe4a6393040/users/8debab7fc32a47839138ef61ef8d8ec9/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:49.374193 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99ff28-8d12-426c-8e10-aaf628015e8b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:49.374773 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99ff28-8d12-426c-8e10-aaf628015e8b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:49.375147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99ff28-8d12-426c-8e10-aaf628015e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:49.468533 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ec99ff28-8d12-426c-8e10-aaf628015e8b None None] MFA Rules not processed for user `8debab7fc32a47839138ef61ef8d8ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:49.470210 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ec99ff28-8d12-426c-8e10-aaf628015e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:49.710671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 882/1756] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:49 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:44:49.765548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82efa3-20d4-4aee-bfbc-9ec51fc9caca admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:49.768449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7a82efa3-20d4-4aee-bfbc-9ec51fc9caca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:49.846787 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82efa3-20d4-4aee-bfbc-9ec51fc9caca admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:49.853258 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d7ee5-b83e-4f1c-a5eb-7d06db9d3da2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:49.853898 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d7ee5-b83e-4f1c-a5eb-7d06db9d3da2 service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:49.854168 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d7ee5-b83e-4f1c-a5eb-7d06db9d3da2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:49.854436 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d7ee5-b83e-4f1c-a5eb-7d06db9d3da2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:49.866550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d7ee5-b83e-4f1c-a5eb-7d06db9d3da2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:49.868056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d7ee5-b83e-4f1c-a5eb-7d06db9d3da2 service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:49.879731 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 875/1757] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:50.499020 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99ff28-8d12-426c-8e10-aaf628015e8b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:50.511988 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99ff28-8d12-426c-8e10-aaf628015e8b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:50.520074 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755fc27-1eae-4942-93e7-6c34bb44ece7 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8debab7fc32a47839138ef61ef8d8ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b337c4d786e4a5aaec8afe4a6393040', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:50.520732 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755fc27-1eae-4942-93e7-6c34bb44ece7 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:50.521007 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755fc27-1eae-4942-93e7-6c34bb44ece7 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:50.521296 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755fc27-1eae-4942-93e7-6c34bb44ece7 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:50.522013 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755fc27-1eae-4942-93e7-6c34bb44ece7 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:44:50.522312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755fc27-1eae-4942-93e7-6c34bb44ece7 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:50.525302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755fc27-1eae-4942-93e7-6c34bb44ece7 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:50.544438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 883/1758] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:44:50.547851 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d7ee5-b83e-4f1c-a5eb-7d06db9d3da2 service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:50.559769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d7ee5-b83e-4f1c-a5eb-7d06db9d3da2 service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:50.572567 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972559c-c7ad-4d15-b9ac-284335d4ce6c tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8debab7fc32a47839138ef61ef8d8ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b337c4d786e4a5aaec8afe4a6393040', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:50.574150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972559c-c7ad-4d15-b9ac-284335d4ce6c tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:50.575664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972559c-c7ad-4d15-b9ac-284335d4ce6c tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:50.576831 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972559c-c7ad-4d15-b9ac-284335d4ce6c tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:50.577994 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972559c-c7ad-4d15-b9ac-284335d4ce6c tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:50.583539 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972559c-c7ad-4d15-b9ac-284335d4ce6c tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:50.647483 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 876/1759] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:50 2022] POST /identity/v3/projects => generated 354 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:44:50.653462 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755fc27-1eae-4942-93e7-6c34bb44ece7 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:50.668818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755fc27-1eae-4942-93e7-6c34bb44ece7 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:50.759101 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d07ebc-4898-476d-8c90-368abd3c9fbd tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8debab7fc32a47839138ef61ef8d8ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b337c4d786e4a5aaec8afe4a6393040', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:50.759101 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d07ebc-4898-476d-8c90-368abd3c9fbd tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:50.759101 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d07ebc-4898-476d-8c90-368abd3c9fbd tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:50.760075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d07ebc-4898-476d-8c90-368abd3c9fbd tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:50.760075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d07ebc-4898-476d-8c90-368abd3c9fbd tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:44:50.760075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d07ebc-4898-476d-8c90-368abd3c9fbd tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:50.761415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d07ebc-4898-476d-8c90-368abd3c9fbd tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:50.784398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 884/1760] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:44:50.799554 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972559c-c7ad-4d15-b9ac-284335d4ce6c tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:50.870992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d07ebc-4898-476d-8c90-368abd3c9fbd tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:50.891433 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d07ebc-4898-476d-8c90-368abd3c9fbd tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:50.903791 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0177b-1e60-42b0-b8d2-8b10366bf9f6 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8debab7fc32a47839138ef61ef8d8ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b337c4d786e4a5aaec8afe4a6393040', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:50.904132 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0177b-1e60-42b0-b8d2-8b10366bf9f6 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:50.904602 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0177b-1e60-42b0-b8d2-8b10366bf9f6 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:50.905074 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0177b-1e60-42b0-b8d2-8b10366bf9f6 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] PATH_INFO: `/v3/projects/afb50af46c7d4acba45170e7b26cab99` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:50.907630 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:50.907937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a0177b-1e60-42b0-b8d2-8b10366bf9f6 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] RBAC: Authorizing `identity:delete_project(project_id=afb50af46c7d4acba45170e7b26cab99)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:50.912879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a0177b-1e60-42b0-b8d2-8b10366bf9f6 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:50.951943 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972559c-c7ad-4d15-b9ac-284335d4ce6c tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:50.969091 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966aa8d1-c098-4fcc-a437-53b9a5b52fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:50.969868 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966aa8d1-c098-4fcc-a437-53b9a5b52fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:50.970238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966aa8d1-c098-4fcc-a437-53b9a5b52fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:50.970553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966aa8d1-c098-4fcc-a437-53b9a5b52fc9 admin admin] PATH_INFO: `/v3/users/74166f0a6a754cd6b08d2aab14a3cf4e` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:50.972010 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966aa8d1-c098-4fcc-a437-53b9a5b52fc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74166f0a6a754cd6b08d2aab14a3cf4e)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:50.974067 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966aa8d1-c098-4fcc-a437-53b9a5b52fc9 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:50.981012 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:51.013322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-87a0177b-1e60-42b0-b8d2-8b10366bf9f6 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb50af46c7d4acba45170e7b26cab99', 'request_id': 'req-87a0177b-1e60-42b0-b8d2-8b10366bf9f6'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:51.025581 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-87a0177b-1e60-42b0-b8d2-8b10366bf9f6 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb50af46c7d4acba45170e7b26cab99', 'request_id': 'req-87a0177b-1e60-42b0-b8d2-8b10366bf9f6'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:51.035130 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-87a0177b-1e60-42b0-b8d2-8b10366bf9f6 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb50af46c7d4acba45170e7b26cab99', 'request_id': 'req-87a0177b-1e60-42b0-b8d2-8b10366bf9f6'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:51.050008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-87a0177b-1e60-42b0-b8d2-8b10366bf9f6 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] The token cache is being invalidate because project afb50af46c7d4acba45170e7b26cab99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:51.050609 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-87a0177b-1e60-42b0-b8d2-8b10366bf9f6 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:51.051462 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-87a0177b-1e60-42b0-b8d2-8b10366bf9f6 tempest-VolumeQuotaClassesTest-961874164 tempest-VolumeQuotaClassesTest-961874164-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:51.053577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 885/1761] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:50 2022] DELETE /identity/v3/projects/afb50af46c7d4acba45170e7b26cab99 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:51.060409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-966aa8d1-c098-4fcc-a437-53b9a5b52fc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74166f0a6a754cd6b08d2aab14a3cf4e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:51.061305 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-966aa8d1-c098-4fcc-a437-53b9a5b52fc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74166f0a6a754cd6b08d2aab14a3cf4e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:51.062374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-966aa8d1-c098-4fcc-a437-53b9a5b52fc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74166f0a6a754cd6b08d2aab14a3cf4e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:51.073071 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-966aa8d1-c098-4fcc-a437-53b9a5b52fc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74166f0a6a754cd6b08d2aab14a3cf4e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:51.093389 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-966aa8d1-c098-4fcc-a437-53b9a5b52fc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74166f0a6a754cd6b08d2aab14a3cf4e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:51.116932 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 877/1762] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:50 2022] DELETE /identity/v3/users/74166f0a6a754cd6b08d2aab14a3cf4e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:51.122158 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd3cc9-14e2-47ec-a31c-106215132f7f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:51.122566 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd3cc9-14e2-47ec-a31c-106215132f7f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:51.123051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd3cc9-14e2-47ec-a31c-106215132f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:51.153232 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-bcbd3cc9-14e2-47ec-a31c-106215132f7f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:51.155640 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bcbd3cc9-14e2-47ec-a31c-106215132f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:51.245384 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 886/1763] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:51.255132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966aa8d1-c098-4fcc-a437-53b9a5b52fc9 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:51.257395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-966aa8d1-c098-4fcc-a437-53b9a5b52fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:51.336365 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966aa8d1-c098-4fcc-a437-53b9a5b52fc9 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:51.341653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a3708-61d4-4c71-b88d-044e209451cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:51.343241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a3708-61d4-4c71-b88d-044e209451cc service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:51.343696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a3708-61d4-4c71-b88d-044e209451cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:51.344187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a3708-61d4-4c71-b88d-044e209451cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:51.362270 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a3708-61d4-4c71-b88d-044e209451cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:51.364559 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a3708-61d4-4c71-b88d-044e209451cc service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:51.384100 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 878/1764] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:44:51.396831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd3cc9-14e2-47ec-a31c-106215132f7f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:51.399498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bcbd3cc9-14e2-47ec-a31c-106215132f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:51.422807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd3cc9-14e2-47ec-a31c-106215132f7f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:51.432141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecadfb0-5f89-4ccd-b606-81e694a60796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:51.433440 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecadfb0-5f89-4ccd-b606-81e694a60796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:51.433909 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecadfb0-5f89-4ccd-b606-81e694a60796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:51.434394 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecadfb0-5f89-4ccd-b606-81e694a60796 admin admin] PATH_INFO: `/v3/users/b9dc342fbc6a47b8a02c8172c77f7974` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:51.437486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecadfb0-5f89-4ccd-b606-81e694a60796 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9dc342fbc6a47b8a02c8172c77f7974)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:51.442037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecadfb0-5f89-4ccd-b606-81e694a60796 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:51.536922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6ecadfb0-5f89-4ccd-b606-81e694a60796 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9dc342fbc6a47b8a02c8172c77f7974'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:51.546913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6ecadfb0-5f89-4ccd-b606-81e694a60796 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9dc342fbc6a47b8a02c8172c77f7974'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:51.557383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6ecadfb0-5f89-4ccd-b606-81e694a60796 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9dc342fbc6a47b8a02c8172c77f7974'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:51.558340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6ecadfb0-5f89-4ccd-b606-81e694a60796 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9dc342fbc6a47b8a02c8172c77f7974'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:51.577811 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6ecadfb0-5f89-4ccd-b606-81e694a60796 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9dc342fbc6a47b8a02c8172c77f7974'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:51.586066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 887/1765] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:51 2022] DELETE /identity/v3/users/b9dc342fbc6a47b8a02c8172c77f7974 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:51.592069 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28f658-11ef-46e3-97c5-960fd84df82a None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:51.592391 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28f658-11ef-46e3-97c5-960fd84df82a None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:51.592650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28f658-11ef-46e3-97c5-960fd84df82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:51.618165 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-4b28f658-11ef-46e3-97c5-960fd84df82a None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:51.620064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4b28f658-11ef-46e3-97c5-960fd84df82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:51.718563 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 879/1766] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:44:51.720599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecadfb0-5f89-4ccd-b606-81e694a60796 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:51.723025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6ecadfb0-5f89-4ccd-b606-81e694a60796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:51.778759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:51.818758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecadfb0-5f89-4ccd-b606-81e694a60796 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:51.822956 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977274d-8528-4758-b5dd-34f1c3110881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:51.823604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977274d-8528-4758-b5dd-34f1c3110881 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:51.823877 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977274d-8528-4758-b5dd-34f1c3110881 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:51.824203 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977274d-8528-4758-b5dd-34f1c3110881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:51.837292 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7977274d-8528-4758-b5dd-34f1c3110881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:51.839599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7977274d-8528-4758-b5dd-34f1c3110881 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:51.860733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 888/1767] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:51.907911 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28f658-11ef-46e3-97c5-960fd84df82a None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:51.910527 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4b28f658-11ef-46e3-97c5-960fd84df82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:51.927643 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:51.933814 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28f658-11ef-46e3-97c5-960fd84df82a None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:51.941550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:51.942847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:51.943677 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:51.944192 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0 admin admin] PATH_INFO: `/v3/projects/07d9f3b730824b389fbd3707b805e25c` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:51.946304 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07d9f3b730824b389fbd3707b805e25c)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:51.949294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:52.065586 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07d9f3b730824b389fbd3707b805e25c', 'request_id': 'req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:52.079686 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07d9f3b730824b389fbd3707b805e25c', 'request_id': 'req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:52.109621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07d9f3b730824b389fbd3707b805e25c', 'request_id': 'req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:52.145211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0 admin admin] The token cache is being invalidate because project 07d9f3b730824b389fbd3707b805e25c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:52.145571 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:52.146327 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:52.149518 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 880/1768] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:51 2022] DELETE /identity/v3/projects/07d9f3b730824b389fbd3707b805e25c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:44:52.154354 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977274d-8528-4758-b5dd-34f1c3110881 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:52.167116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977274d-8528-4758-b5dd-34f1c3110881 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:52.293175 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9591ed-c16e-41eb-9b02-65359e866343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:52.294357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9591ed-c16e-41eb-9b02-65359e866343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:52.295357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9591ed-c16e-41eb-9b02-65359e866343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:52.295879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9591ed-c16e-41eb-9b02-65359e866343 admin admin] PATH_INFO: `/v3/users/de28c6bf20b9400d8970e06c963e04a6` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:52.298047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9591ed-c16e-41eb-9b02-65359e866343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de28c6bf20b9400d8970e06c963e04a6)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:52.303095 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9591ed-c16e-41eb-9b02-65359e866343 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:52.413421 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ff9591ed-c16e-41eb-9b02-65359e866343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de28c6bf20b9400d8970e06c963e04a6'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:52.431593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ff9591ed-c16e-41eb-9b02-65359e866343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de28c6bf20b9400d8970e06c963e04a6'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:52.446376 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ff9591ed-c16e-41eb-9b02-65359e866343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de28c6bf20b9400d8970e06c963e04a6'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:52.448262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ff9591ed-c16e-41eb-9b02-65359e866343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de28c6bf20b9400d8970e06c963e04a6'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:52.475741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:52.477659 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:52.487699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ff9591ed-c16e-41eb-9b02-65359e866343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de28c6bf20b9400d8970e06c963e04a6'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:52.504839 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6e287-6aa5-4db9-b61d-8258646f4ef0 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:52.505163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 889/1769] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:52 2022] DELETE /identity/v3/users/de28c6bf20b9400d8970e06c963e04a6 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:52.639573 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27827c96-ab2a-472e-a930-2e7c1112d249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:52.640289 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27827c96-ab2a-472e-a930-2e7c1112d249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:52.640628 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27827c96-ab2a-472e-a930-2e7c1112d249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:52.640925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27827c96-ab2a-472e-a930-2e7c1112d249 admin admin] PATH_INFO: `/v3/projects/93ca9c31302141c3bcece7cc90855c26` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:52.642284 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27827c96-ab2a-472e-a930-2e7c1112d249 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93ca9c31302141c3bcece7cc90855c26)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:52.644683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27827c96-ab2a-472e-a930-2e7c1112d249 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:52.751653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-27827c96-ab2a-472e-a930-2e7c1112d249 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93ca9c31302141c3bcece7cc90855c26', 'request_id': 'req-27827c96-ab2a-472e-a930-2e7c1112d249'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:52.762233 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-27827c96-ab2a-472e-a930-2e7c1112d249 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93ca9c31302141c3bcece7cc90855c26', 'request_id': 'req-27827c96-ab2a-472e-a930-2e7c1112d249'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:52.773110 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-27827c96-ab2a-472e-a930-2e7c1112d249 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93ca9c31302141c3bcece7cc90855c26', 'request_id': 'req-27827c96-ab2a-472e-a930-2e7c1112d249'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:52.791341 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-27827c96-ab2a-472e-a930-2e7c1112d249 admin admin] The token cache is being invalidate because project 93ca9c31302141c3bcece7cc90855c26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:52.791639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-27827c96-ab2a-472e-a930-2e7c1112d249 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:52.792375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-27827c96-ab2a-472e-a930-2e7c1112d249 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:52.793987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 881/1770] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:52 2022] DELETE /identity/v3/projects/93ca9c31302141c3bcece7cc90855c26 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:44:52.798783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9591ed-c16e-41eb-9b02-65359e866343 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:52.800700 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ff9591ed-c16e-41eb-9b02-65359e866343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:52.815297 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9591ed-c16e-41eb-9b02-65359e866343 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:52.885911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:52.925897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd914f-5679-490a-b868-a3415af87f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:52.930039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd914f-5679-490a-b868-a3415af87f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:52.931941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd914f-5679-490a-b868-a3415af87f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:52.937255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd914f-5679-490a-b868-a3415af87f34 admin admin] PATH_INFO: `/v3/users/8debab7fc32a47839138ef61ef8d8ec9` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:52.938323 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd914f-5679-490a-b868-a3415af87f34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8debab7fc32a47839138ef61ef8d8ec9)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:52.940678 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd914f-5679-490a-b868-a3415af87f34 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:53.039215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7fdd914f-5679-490a-b868-a3415af87f34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8debab7fc32a47839138ef61ef8d8ec9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:53.050989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7fdd914f-5679-490a-b868-a3415af87f34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8debab7fc32a47839138ef61ef8d8ec9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:53.067572 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7fdd914f-5679-490a-b868-a3415af87f34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8debab7fc32a47839138ef61ef8d8ec9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:53.068952 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7fdd914f-5679-490a-b868-a3415af87f34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8debab7fc32a47839138ef61ef8d8ec9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:53.104678 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7fdd914f-5679-490a-b868-a3415af87f34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8debab7fc32a47839138ef61ef8d8ec9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:53.107010 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27827c96-ab2a-472e-a930-2e7c1112d249 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:53.109662 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-27827c96-ab2a-472e-a930-2e7c1112d249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:53.113973 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 890/1771] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:52 2022] DELETE /identity/v3/users/8debab7fc32a47839138ef61ef8d8ec9 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:53.134529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27827c96-ab2a-472e-a930-2e7c1112d249 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:53.170801 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:53.377359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63276c36-4ac4-4684-b425-b8896a1c3232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:53.379064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63276c36-4ac4-4684-b425-b8896a1c3232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:53.379620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63276c36-4ac4-4684-b425-b8896a1c3232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:53.380213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63276c36-4ac4-4684-b425-b8896a1c3232 admin admin] PATH_INFO: `/v3/projects/06b30445a1af40239a9360cd6c8bebb6` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:53.382681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63276c36-4ac4-4684-b425-b8896a1c3232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06b30445a1af40239a9360cd6c8bebb6)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:53.386231 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63276c36-4ac4-4684-b425-b8896a1c3232 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:53.683002 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-63276c36-4ac4-4684-b425-b8896a1c3232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06b30445a1af40239a9360cd6c8bebb6', 'request_id': 'req-63276c36-4ac4-4684-b425-b8896a1c3232'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:53.715231 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-63276c36-4ac4-4684-b425-b8896a1c3232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06b30445a1af40239a9360cd6c8bebb6', 'request_id': 'req-63276c36-4ac4-4684-b425-b8896a1c3232'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:53.749916 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-63276c36-4ac4-4684-b425-b8896a1c3232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06b30445a1af40239a9360cd6c8bebb6', 'request_id': 'req-63276c36-4ac4-4684-b425-b8896a1c3232'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:53.796903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd914f-5679-490a-b868-a3415af87f34 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:53.801457 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7fdd914f-5679-490a-b868-a3415af87f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:53.837340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd914f-5679-490a-b868-a3415af87f34 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:53.837675 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-63276c36-4ac4-4684-b425-b8896a1c3232 admin admin] The token cache is being invalidate because project 06b30445a1af40239a9360cd6c8bebb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:53.848643 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-63276c36-4ac4-4684-b425-b8896a1c3232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:53.852120 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-63276c36-4ac4-4684-b425-b8896a1c3232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:53.864837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 882/1772] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:53 2022] DELETE /identity/v3/projects/06b30445a1af40239a9360cd6c8bebb6 => generated 0 bytes in 759 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:44:53.874945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551b2e6-fb63-4a41-b49f-b2a1f2327c54 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:53.875526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551b2e6-fb63-4a41-b49f-b2a1f2327c54 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:53.876152 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551b2e6-fb63-4a41-b49f-b2a1f2327c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:53.999672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-f551b2e6-fb63-4a41-b49f-b2a1f2327c54 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:54.002198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f551b2e6-fb63-4a41-b49f-b2a1f2327c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:54.092070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:54.092818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:54.094717 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:54.096491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f admin admin] PATH_INFO: `/v3/projects/1b337c4d786e4a5aaec8afe4a6393040` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:54.099475 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b337c4d786e4a5aaec8afe4a6393040)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:54.104509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:54.159682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 883/1773] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:54.168712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6304acd-4e17-484b-b1bc-8677c9a0ff48 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:54.169020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6304acd-4e17-484b-b1bc-8677c9a0ff48 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:54.169432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6304acd-4e17-484b-b1bc-8677c9a0ff48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:54.234101 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b337c4d786e4a5aaec8afe4a6393040', 'request_id': 'req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:54.236681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-f6304acd-4e17-484b-b1bc-8677c9a0ff48 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:54.238797 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f6304acd-4e17-484b-b1bc-8677c9a0ff48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:54.250028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b337c4d786e4a5aaec8afe4a6393040', 'request_id': 'req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:54.262997 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b337c4d786e4a5aaec8afe4a6393040', 'request_id': 'req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:54.275702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:54.282597 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f admin admin] The token cache is being invalidate because project 1b337c4d786e4a5aaec8afe4a6393040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:54.285487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:54.287276 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:54.289752 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 891/1774] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:53 2022] DELETE /identity/v3/projects/1b337c4d786e4a5aaec8afe4a6393040 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:54.290796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:54.296930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:54.344415 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:54.393593 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 884/1775] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:54.396075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44c36b-fdfc-49a6-9653-10de8d42c245 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:54.396658 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44c36b-fdfc-49a6-9653-10de8d42c245 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:54.397288 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44c36b-fdfc-49a6-9653-10de8d42c245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:54.433791 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-2e44c36b-fdfc-49a6-9653-10de8d42c245 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:54.435671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2e44c36b-fdfc-49a6-9653-10de8d42c245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:54.447876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 885/1776] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:54.449766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44c36b-fdfc-49a6-9653-10de8d42c245 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:54.454464 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e6ca0-95bc-43f3-86f0-57af24fd4d0f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:54.461731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233e047-c483-4ada-a990-89d02151ecb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:54.462577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233e047-c483-4ada-a990-89d02151ecb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:54.462950 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233e047-c483-4ada-a990-89d02151ecb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:54.463424 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233e047-c483-4ada-a990-89d02151ecb4 admin admin] PATH_INFO: `/v3/users/14fe67a054de49a9af8e240ef5187db5` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:54.467123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233e047-c483-4ada-a990-89d02151ecb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14fe67a054de49a9af8e240ef5187db5)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:54.470660 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233e047-c483-4ada-a990-89d02151ecb4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:54.471189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44c36b-fdfc-49a6-9653-10de8d42c245 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:54.478929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e9263-1b30-4427-ac83-72274c7803f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:54.479946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e9263-1b30-4427-ac83-72274c7803f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:54.480419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e9263-1b30-4427-ac83-72274c7803f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:54.480902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e9263-1b30-4427-ac83-72274c7803f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:54.482195 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e9263-1b30-4427-ac83-72274c7803f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:44:54.483061 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e9263-1b30-4427-ac83-72274c7803f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:54.485343 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e9263-1b30-4427-ac83-72274c7803f5 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:54.501820 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 886/1777] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:44:54.526094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2efff-d9ba-4223-8cc8-9a9006e049c6 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:54.526807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2efff-d9ba-4223-8cc8-9a9006e049c6 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:54.527875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2efff-d9ba-4223-8cc8-9a9006e049c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:54.585969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-d6b2efff-d9ba-4223-8cc8-9a9006e049c6 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:54.593448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d6b2efff-d9ba-4223-8cc8-9a9006e049c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:54.612198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 887/1778] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:54.613430 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2efff-d9ba-4223-8cc8-9a9006e049c6 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:54.632655 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e233e047-c483-4ada-a990-89d02151ecb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14fe67a054de49a9af8e240ef5187db5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:54.657446 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e233e047-c483-4ada-a990-89d02151ecb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14fe67a054de49a9af8e240ef5187db5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:54.670778 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2efff-d9ba-4223-8cc8-9a9006e049c6 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:54.688174 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e233e047-c483-4ada-a990-89d02151ecb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14fe67a054de49a9af8e240ef5187db5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:54.692230 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ebc9c-819d-42cd-a037-410b3323b657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:54.699539 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e233e047-c483-4ada-a990-89d02151ecb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14fe67a054de49a9af8e240ef5187db5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:54.701371 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ebc9c-819d-42cd-a037-410b3323b657 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:54.702303 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ebc9c-819d-42cd-a037-410b3323b657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:54.703775 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ebc9c-819d-42cd-a037-410b3323b657 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:54.708928 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ebc9c-819d-42cd-a037-410b3323b657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:54.721580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ebc9c-819d-42cd-a037-410b3323b657 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:54.781804 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e233e047-c483-4ada-a990-89d02151ecb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14fe67a054de49a9af8e240ef5187db5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:54.813879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 892/1779] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:54 2022] DELETE /identity/v3/users/14fe67a054de49a9af8e240ef5187db5 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:54.814947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233e047-c483-4ada-a990-89d02151ecb4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:54.827594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e233e047-c483-4ada-a990-89d02151ecb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:54.832881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 888/1780] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:54 2022] POST /identity/v3/projects => generated 383 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:54.838182 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d96192-f6b0-42b2-9d94-d087edb162c0 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:54.838427 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d96192-f6b0-42b2-9d94-d087edb162c0 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:54.838738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d96192-f6b0-42b2-9d94-d087edb162c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:54.889752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-d8d96192-f6b0-42b2-9d94-d087edb162c0 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:54.891140 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d8d96192-f6b0-42b2-9d94-d087edb162c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:55.099696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 889/1781] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:55.101737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d96192-f6b0-42b2-9d94-d087edb162c0 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:55.104588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d8d96192-f6b0-42b2-9d94-d087edb162c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:55.144072 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d96192-f6b0-42b2-9d94-d087edb162c0 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:55.146073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233e047-c483-4ada-a990-89d02151ecb4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:55.163478 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d287e34-a3b2-4f96-ab6e-66857237160b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:55.165380 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d287e34-a3b2-4f96-ab6e-66857237160b admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:55.165891 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d287e34-a3b2-4f96-ab6e-66857237160b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:55.166370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d287e34-a3b2-4f96-ab6e-66857237160b admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:55.167280 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d287e34-a3b2-4f96-ab6e-66857237160b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:44:55.167762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d287e34-a3b2-4f96-ab6e-66857237160b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:55.169965 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d287e34-a3b2-4f96-ab6e-66857237160b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:55.205599 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbab1b9-5986-42de-b059-08102f733319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:55.206546 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 893/1782] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:44:55.208331 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d287e34-a3b2-4f96-ab6e-66857237160b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:55.213215 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbab1b9-5986-42de-b059-08102f733319 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:55.223454 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbab1b9-5986-42de-b059-08102f733319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:55.227585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbab1b9-5986-42de-b059-08102f733319 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:55.234500 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d287e34-a3b2-4f96-ab6e-66857237160b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:55.236174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbab1b9-5986-42de-b059-08102f733319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:55.251037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7abd4-d5b4-4983-855c-332238bc1eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:55.253507 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7abd4-d5b4-4983-855c-332238bc1eed admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:55.253772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbab1b9-5986-42de-b059-08102f733319 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:55.253929 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7abd4-d5b4-4983-855c-332238bc1eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:55.254094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7abd4-d5b4-4983-855c-332238bc1eed admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:55.255531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7abd4-d5b4-4983-855c-332238bc1eed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:55.256758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7abd4-d5b4-4983-855c-332238bc1eed admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:55.296234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:55.332472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 890/1783] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:44:55 2022] POST /identity/v3/users => generated 292 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:55.339477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbab1b9-5986-42de-b059-08102f733319 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:55.343211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-9bbab1b9-5986-42de-b059-08102f733319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:55.351577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 894/1784] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:55 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:55.352469 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7abd4-d5b4-4983-855c-332238bc1eed admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:55.501962 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbab1b9-5986-42de-b059-08102f733319 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:55.545998 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7abd4-d5b4-4983-855c-332238bc1eed admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:55.586024 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e83020-2229-495f-826d-125e2998d183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:55.587590 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e83020-2229-495f-826d-125e2998d183 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:55.588217 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e83020-2229-495f-826d-125e2998d183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:55.588661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e83020-2229-495f-826d-125e2998d183 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:55.589933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e83020-2229-495f-826d-125e2998d183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:55.602523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e83020-2229-495f-826d-125e2998d183 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:55.657166 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 895/1785] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:55.657166 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e83020-2229-495f-826d-125e2998d183 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:55.694902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:55.695908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e83020-2229-495f-826d-125e2998d183 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:55.721917 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36812d-612f-4884-a9c6-2aba3f0579f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:55.727169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36812d-612f-4884-a9c6-2aba3f0579f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:55.727758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36812d-612f-4884-a9c6-2aba3f0579f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:55.731494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36812d-612f-4884-a9c6-2aba3f0579f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:55.735150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55253340-2e9e-4d09-87a6-8a91285cce10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:55.735857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253340-2e9e-4d09-87a6-8a91285cce10 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:55.736153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253340-2e9e-4d09-87a6-8a91285cce10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:55.736401 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36812d-612f-4884-a9c6-2aba3f0579f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:55.736513 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253340-2e9e-4d09-87a6-8a91285cce10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:55.753756 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36812d-612f-4884-a9c6-2aba3f0579f4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:55.761883 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55253340-2e9e-4d09-87a6-8a91285cce10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:55.764021 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55253340-2e9e-4d09-87a6-8a91285cce10 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:55.804819 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 891/1786] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:55.804819 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55253340-2e9e-4d09-87a6-8a91285cce10 service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:55.825888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55253340-2e9e-4d09-87a6-8a91285cce10 service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:55.833101 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74376c-bf3a-4022-b179-69f4947c28ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:55.833489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74376c-bf3a-4022-b179-69f4947c28ed admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:55.833736 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74376c-bf3a-4022-b179-69f4947c28ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:55.834096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74376c-bf3a-4022-b179-69f4947c28ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:55.834940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74376c-bf3a-4022-b179-69f4947c28ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:55.837015 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74376c-bf3a-4022-b179-69f4947c28ed admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:55.872542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 892/1787] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:55.878880 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74376c-bf3a-4022-b179-69f4947c28ed admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:55.881967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 896/1788] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:44:55 2022] POST /identity/v3/users => generated 291 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:55.898372 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74376c-bf3a-4022-b179-69f4947c28ed admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:55.899302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36812d-612f-4884-a9c6-2aba3f0579f4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:55.913752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94acfeb0-81bd-4b1b-89d2-04e98828793d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:55.914353 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acfeb0-81bd-4b1b-89d2-04e98828793d admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:55.914826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acfeb0-81bd-4b1b-89d2-04e98828793d admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:55.914945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acfeb0-81bd-4b1b-89d2-04e98828793d admin admin] PATH_INFO: `/v3/projects/7ec055a45c5a4a0d9a763daceb8131e9/users/72ec3418de51407fa670e959e288cc64/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:55.920947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36812d-612f-4884-a9c6-2aba3f0579f4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:55.929555 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72c898-e07f-4895-a881-fc5fd8737dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:55.931391 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72c898-e07f-4895-a881-fc5fd8737dff admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:55.932121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72c898-e07f-4895-a881-fc5fd8737dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:55.932995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72c898-e07f-4895-a881-fc5fd8737dff admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:55.934905 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72c898-e07f-4895-a881-fc5fd8737dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:55.938785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72c898-e07f-4895-a881-fc5fd8737dff admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:55.956490 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94acfeb0-81bd-4b1b-89d2-04e98828793d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec055a45c5a4a0d9a763daceb8131e9, user_id=72ec3418de51407fa670e959e288cc64, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:55.959684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94acfeb0-81bd-4b1b-89d2-04e98828793d admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:55.961583 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 897/1789] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:55.966179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72c898-e07f-4895-a881-fc5fd8737dff admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:55.986085 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72c898-e07f-4895-a881-fc5fd8737dff admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:55.991319 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 893/1790] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:55 2022] PUT /identity/v3/projects/7ec055a45c5a4a0d9a763daceb8131e9/users/72ec3418de51407fa670e959e288cc64/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:56.013840 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e113de9-5741-4569-ab9e-a78397496afa None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:56.013840 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e113de9-5741-4569-ab9e-a78397496afa None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:56.013840 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e113de9-5741-4569-ab9e-a78397496afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:56.068371 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-8e113de9-5741-4569-ab9e-a78397496afa None None] MFA Rules not processed for user `72ec3418de51407fa670e959e288cc64`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:56.070422 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-8e113de9-5741-4569-ab9e-a78397496afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:56.129672 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec18d7e-f2f8-4acc-8ae4-9bc1aef0905b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:56.130310 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec18d7e-f2f8-4acc-8ae4-9bc1aef0905b admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:56.133403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec18d7e-f2f8-4acc-8ae4-9bc1aef0905b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:56.133670 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec18d7e-f2f8-4acc-8ae4-9bc1aef0905b admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:56.137424 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec18d7e-f2f8-4acc-8ae4-9bc1aef0905b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:56.140746 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec18d7e-f2f8-4acc-8ae4-9bc1aef0905b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:56.175061 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 898/1791] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:56.176141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec18d7e-f2f8-4acc-8ae4-9bc1aef0905b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:56.191605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec18d7e-f2f8-4acc-8ae4-9bc1aef0905b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:56.197729 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1651c-2920-4994-87e2-0f81ddb3fdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:56.198488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1651c-2920-4994-87e2-0f81ddb3fdf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:56.198807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1651c-2920-4994-87e2-0f81ddb3fdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:56.199063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1651c-2920-4994-87e2-0f81ddb3fdf7 admin admin] PATH_INFO: `/v3/projects/902ad629e7e54ba0bef88ca561dcb819/users/6068bc03b9014132bf7a9b85c9f48c58/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:56.233317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1651c-2920-4994-87e2-0f81ddb3fdf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=902ad629e7e54ba0bef88ca561dcb819, user_id=6068bc03b9014132bf7a9b85c9f48c58, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:56.236821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1651c-2920-4994-87e2-0f81ddb3fdf7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:56.256640 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 899/1792] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:56 2022] PUT /identity/v3/projects/902ad629e7e54ba0bef88ca561dcb819/users/6068bc03b9014132bf7a9b85c9f48c58/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:56.260900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 894/1793] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:56.274908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1651c-2920-4994-87e2-0f81ddb3fdf7 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:56.280545 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20997c6-182d-4040-a92e-971d55b8261b None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:56.281102 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20997c6-182d-4040-a92e-971d55b8261b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:56.281395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20997c6-182d-4040-a92e-971d55b8261b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:56.319915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a20997c6-182d-4040-a92e-971d55b8261b None None] MFA Rules not processed for user `6068bc03b9014132bf7a9b85c9f48c58`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:56.321737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a20997c6-182d-4040-a92e-971d55b8261b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:56.322789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:56.365707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1651c-2920-4994-87e2-0f81ddb3fdf7 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:56.372173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6fb985-9f73-418f-9c24-4b43886b09dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:56.373093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fb985-9f73-418f-9c24-4b43886b09dc admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:56.373519 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fb985-9f73-418f-9c24-4b43886b09dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:56.373792 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fb985-9f73-418f-9c24-4b43886b09dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:56.375012 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6fb985-9f73-418f-9c24-4b43886b09dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:56.377433 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6fb985-9f73-418f-9c24-4b43886b09dc admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:56.441946 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 900/1794] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:56 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:56.447584 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6fb985-9f73-418f-9c24-4b43886b09dc admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:56.475855 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 895/1795] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:56.477917 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20997c6-182d-4040-a92e-971d55b8261b None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:56.500028 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20997c6-182d-4040-a92e-971d55b8261b None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:56.541653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:56.552990 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6fb985-9f73-418f-9c24-4b43886b09dc admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:56.560660 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f24a7-0e35-4de4-b1d1-44ebd5850998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:56.562167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f24a7-0e35-4de4-b1d1-44ebd5850998 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:56.562662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f24a7-0e35-4de4-b1d1-44ebd5850998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:56.563164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f24a7-0e35-4de4-b1d1-44ebd5850998 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:56.564683 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f24a7-0e35-4de4-b1d1-44ebd5850998 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:56.568372 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f24a7-0e35-4de4-b1d1-44ebd5850998 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:56.613604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 901/1796] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:44:56 2022] POST /identity/v3/users => generated 299 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:44:56.619507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa94f27-181c-4177-9576-1c35db432424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:56.620794 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa94f27-181c-4177-9576-1c35db432424 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:56.621202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa94f27-181c-4177-9576-1c35db432424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:56.621579 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa94f27-181c-4177-9576-1c35db432424 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:56.622404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa94f27-181c-4177-9576-1c35db432424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:56.627233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f24a7-0e35-4de4-b1d1-44ebd5850998 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:56.635497 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa94f27-181c-4177-9576-1c35db432424 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:56.667707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f24a7-0e35-4de4-b1d1-44ebd5850998 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:56.676308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4313de-2710-4490-866a-8343bff6f2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:56.678757 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4313de-2710-4490-866a-8343bff6f2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:56.679431 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4313de-2710-4490-866a-8343bff6f2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:56.680587 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4313de-2710-4490-866a-8343bff6f2a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:56.681598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4313de-2710-4490-866a-8343bff6f2a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:56.684558 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4313de-2710-4490-866a-8343bff6f2a5 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:56.716059 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 896/1797] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:56 2022] POST /identity/v3/projects => generated 383 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:44:56.726403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 902/1798] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:56.733172 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4313de-2710-4490-866a-8343bff6f2a5 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:56.733758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa94f27-181c-4177-9576-1c35db432424 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:56.760312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4313de-2710-4490-866a-8343bff6f2a5 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:56.831823 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa94f27-181c-4177-9576-1c35db432424 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:56.838427 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc3083-6c0b-4b50-a74e-cf346282aaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:56.839227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc3083-6c0b-4b50-a74e-cf346282aaaa admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:56.839609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc3083-6c0b-4b50-a74e-cf346282aaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:56.840137 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc3083-6c0b-4b50-a74e-cf346282aaaa admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:56.840960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc3083-6c0b-4b50-a74e-cf346282aaaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:56.844247 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc3083-6c0b-4b50-a74e-cf346282aaaa admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:56.890754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 897/1799] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:44:56 2022] POST /identity/v3/users => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:56.893330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc3083-6c0b-4b50-a74e-cf346282aaaa admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:56.914670 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc3083-6c0b-4b50-a74e-cf346282aaaa admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:56.928803 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2aa31-c087-430f-88e4-038c5e4553f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:56.931271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2aa31-c087-430f-88e4-038c5e4553f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:56.932538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2aa31-c087-430f-88e4-038c5e4553f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:56.934018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f981116-612e-41f7-afa1-62bce9e60492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:56.934886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2aa31-c087-430f-88e4-038c5e4553f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:56.935749 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2aa31-c087-430f-88e4-038c5e4553f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:56.936081 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f981116-612e-41f7-afa1-62bce9e60492 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:56.937545 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f981116-612e-41f7-afa1-62bce9e60492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:56.937885 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f981116-612e-41f7-afa1-62bce9e60492 admin admin] PATH_INFO: `/v3/projects/e4e68229966146958100b8903057e63e/users/c7d59b2f5c944b92b37360ecc88d84ca/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:56.941690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2aa31-c087-430f-88e4-038c5e4553f8 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:56.970423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 898/1800] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:56.974434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2aa31-c087-430f-88e4-038c5e4553f8 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:56.974642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f981116-612e-41f7-afa1-62bce9e60492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4e68229966146958100b8903057e63e, user_id=c7d59b2f5c944b92b37360ecc88d84ca, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:56.977941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f981116-612e-41f7-afa1-62bce9e60492 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:56.988041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2aa31-c087-430f-88e4-038c5e4553f8 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:56.995787 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04c3d0-6f96-4f33-a11a-f846a96ef42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:56.996693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04c3d0-6f96-4f33-a11a-f846a96ef42e admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:56.997017 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04c3d0-6f96-4f33-a11a-f846a96ef42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:56.997293 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04c3d0-6f96-4f33-a11a-f846a96ef42e admin admin] PATH_INFO: `/v3/projects/6b193466c8384de5a9c7152d5c584ec4/users/809a0d8eb0eb4efda036129a65cc729a/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:57.001871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 903/1801] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:56 2022] PUT /identity/v3/projects/e4e68229966146958100b8903057e63e/users/c7d59b2f5c944b92b37360ecc88d84ca/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:57.006379 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f981116-612e-41f7-afa1-62bce9e60492 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:57.029581 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:57.034529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04c3d0-6f96-4f33-a11a-f846a96ef42e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b193466c8384de5a9c7152d5c584ec4, user_id=809a0d8eb0eb4efda036129a65cc729a, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:57.037782 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04c3d0-6f96-4f33-a11a-f846a96ef42e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:57.074150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 899/1802] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:56 2022] PUT /identity/v3/projects/6b193466c8384de5a9c7152d5c584ec4/users/809a0d8eb0eb4efda036129a65cc729a/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:57.074150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04c3d0-6f96-4f33-a11a-f846a96ef42e admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:57.081526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-eb04c3d0-6f96-4f33-a11a-f846a96ef42e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:57.162737 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f981116-612e-41f7-afa1-62bce9e60492 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:57.244213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04c3d0-6f96-4f33-a11a-f846a96ef42e admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:57.259408 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18110159-2d6a-4ead-9bfe-f22918029daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:57.261287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18110159-2d6a-4ead-9bfe-f22918029daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:57.262181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18110159-2d6a-4ead-9bfe-f22918029daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:57.262181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18110159-2d6a-4ead-9bfe-f22918029daa admin admin] PATH_INFO: `/v3/projects/90aefd1c4ec24087a93eb07b4a62c94e` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:57.267720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18110159-2d6a-4ead-9bfe-f22918029daa admin admin] RBAC: Authorizing `identity:delete_project(project_id=90aefd1c4ec24087a93eb07b4a62c94e)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:57.269822 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18110159-2d6a-4ead-9bfe-f22918029daa admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:57.283868 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:57.306196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45126f-5d30-4db5-af85-0bca842bcb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:57.306841 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45126f-5d30-4db5-af85-0bca842bcb6c admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:57.307128 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45126f-5d30-4db5-af85-0bca842bcb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:57.307416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45126f-5d30-4db5-af85-0bca842bcb6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:57.308123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45126f-5d30-4db5-af85-0bca842bcb6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:57.310187 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45126f-5d30-4db5-af85-0bca842bcb6c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:57.330598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 904/1803] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:44:57.331665 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45126f-5d30-4db5-af85-0bca842bcb6c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:57.346398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45126f-5d30-4db5-af85-0bca842bcb6c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:57.352505 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29b981-c88e-489e-9d59-825e55398466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:57.353168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29b981-c88e-489e-9d59-825e55398466 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:57.353458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29b981-c88e-489e-9d59-825e55398466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:57.353727 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29b981-c88e-489e-9d59-825e55398466 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:57.354404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29b981-c88e-489e-9d59-825e55398466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:57.359008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29b981-c88e-489e-9d59-825e55398466 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:57.381533 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 905/1804] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:57.382685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29b981-c88e-489e-9d59-825e55398466 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:57.396611 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29b981-c88e-489e-9d59-825e55398466 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:57.449738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-18110159-2d6a-4ead-9bfe-f22918029daa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90aefd1c4ec24087a93eb07b4a62c94e', 'request_id': 'req-18110159-2d6a-4ead-9bfe-f22918029daa'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:57.482012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-18110159-2d6a-4ead-9bfe-f22918029daa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90aefd1c4ec24087a93eb07b4a62c94e', 'request_id': 'req-18110159-2d6a-4ead-9bfe-f22918029daa'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:57.489459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775c32a-15c4-477a-9af1-a8bf964dbf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:57.490535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775c32a-15c4-477a-9af1-a8bf964dbf9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:57.490821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775c32a-15c4-477a-9af1-a8bf964dbf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:57.491105 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775c32a-15c4-477a-9af1-a8bf964dbf9a admin admin] PATH_INFO: `/v3/projects/e4e68229966146958100b8903057e63e/users/c7d59b2f5c944b92b37360ecc88d84ca/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:57.493991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775c32a-15c4-477a-9af1-a8bf964dbf9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4e68229966146958100b8903057e63e, user_id=c7d59b2f5c944b92b37360ecc88d84ca, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:57.496974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775c32a-15c4-477a-9af1-a8bf964dbf9a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:57.499255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-18110159-2d6a-4ead-9bfe-f22918029daa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90aefd1c4ec24087a93eb07b4a62c94e', 'request_id': 'req-18110159-2d6a-4ead-9bfe-f22918029daa'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:57.522987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 906/1805] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:57 2022] PUT /identity/v3/projects/e4e68229966146958100b8903057e63e/users/c7d59b2f5c944b92b37360ecc88d84ca/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:57.524213 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775c32a-15c4-477a-9af1-a8bf964dbf9a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:57.527967 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-18110159-2d6a-4ead-9bfe-f22918029daa admin admin] The token cache is being invalidate because project 90aefd1c4ec24087a93eb07b4a62c94e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:57.528975 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-18110159-2d6a-4ead-9bfe-f22918029daa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:57.532287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-18110159-2d6a-4ead-9bfe-f22918029daa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:57.536304 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 900/1806] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:57 2022] DELETE /identity/v3/projects/90aefd1c4ec24087a93eb07b4a62c94e => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:57.540428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2346-3442-4531-b434-76143b0d3bde None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:57.540889 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2346-3442-4531-b434-76143b0d3bde None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:57.541999 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2346-3442-4531-b434-76143b0d3bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:57.661713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:57.678389 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-6b7f2346-3442-4531-b434-76143b0d3bde None None] MFA Rules not processed for user `c7d59b2f5c944b92b37360ecc88d84ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:57.690238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6b7f2346-3442-4531-b434-76143b0d3bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:57.699888 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775c32a-15c4-477a-9af1-a8bf964dbf9a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:57.710518 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89478033-bee3-4434-b4f2-7bc8986b772d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:57.711742 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89478033-bee3-4434-b4f2-7bc8986b772d admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:57.712358 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89478033-bee3-4434-b4f2-7bc8986b772d admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:57.713265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89478033-bee3-4434-b4f2-7bc8986b772d admin admin] PATH_INFO: `/v3/projects/6b193466c8384de5a9c7152d5c584ec4/users/809a0d8eb0eb4efda036129a65cc729a/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:57.718785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89478033-bee3-4434-b4f2-7bc8986b772d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b193466c8384de5a9c7152d5c584ec4, user_id=809a0d8eb0eb4efda036129a65cc729a, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:57.725352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89478033-bee3-4434-b4f2-7bc8986b772d admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:57.749702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 907/1807] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:57 2022] PUT /identity/v3/projects/6b193466c8384de5a9c7152d5c584ec4/users/809a0d8eb0eb4efda036129a65cc729a/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:57.751057 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89478033-bee3-4434-b4f2-7bc8986b772d admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:57.779740 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-89478033-bee3-4434-b4f2-7bc8986b772d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:57.827638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 901/1808] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:57.828816 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed44ee-a6da-49bb-a574-69886e8cad46 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:57.829103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed44ee-a6da-49bb-a574-69886e8cad46 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:57.829468 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed44ee-a6da-49bb-a574-69886e8cad46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:57.884467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89478033-bee3-4434-b4f2-7bc8986b772d admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:57.890630 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d36e18-6553-4355-8173-3ca2c27d32e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:57.891402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d36e18-6553-4355-8173-3ca2c27d32e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:57.891800 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d36e18-6553-4355-8173-3ca2c27d32e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:57.892654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d36e18-6553-4355-8173-3ca2c27d32e8 admin admin] PATH_INFO: `/v3/users/dd570edc12614c20acaba510ecc9a61e` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:57.893837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d36e18-6553-4355-8173-3ca2c27d32e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd570edc12614c20acaba510ecc9a61e)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:57.896738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d36e18-6553-4355-8173-3ca2c27d32e8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:57.901249 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-c9ed44ee-a6da-49bb-a574-69886e8cad46 None None] MFA Rules not processed for user `809a0d8eb0eb4efda036129a65cc729a`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:57.906620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c9ed44ee-a6da-49bb-a574-69886e8cad46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:57.977385 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-76d36e18-6553-4355-8173-3ca2c27d32e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd570edc12614c20acaba510ecc9a61e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:57.991299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-76d36e18-6553-4355-8173-3ca2c27d32e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd570edc12614c20acaba510ecc9a61e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.006214 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-76d36e18-6553-4355-8173-3ca2c27d32e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd570edc12614c20acaba510ecc9a61e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.007181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-76d36e18-6553-4355-8173-3ca2c27d32e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd570edc12614c20acaba510ecc9a61e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.038801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-76d36e18-6553-4355-8173-3ca2c27d32e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd570edc12614c20acaba510ecc9a61e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.048233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 908/1809] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:57 2022] DELETE /identity/v3/users/dd570edc12614c20acaba510ecc9a61e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:58.057121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d36e18-6553-4355-8173-3ca2c27d32e8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:58.059496 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-76d36e18-6553-4355-8173-3ca2c27d32e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:58.081991 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 902/1810] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:58.093438 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed44ee-a6da-49bb-a574-69886e8cad46 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:58.096241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c9ed44ee-a6da-49bb-a574-69886e8cad46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:58.112941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:58.192189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d36e18-6553-4355-8173-3ca2c27d32e8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:58.197139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48282db2-deb5-4840-974c-6c859836f0f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:58.198396 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48282db2-deb5-4840-974c-6c859836f0f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:58.198855 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48282db2-deb5-4840-974c-6c859836f0f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:58.199431 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48282db2-deb5-4840-974c-6c859836f0f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:58.202134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-48282db2-deb5-4840-974c-6c859836f0f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:58.218503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:58.253742 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed44ee-a6da-49bb-a574-69886e8cad46 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:58.265646 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76994001-a206-43ff-9ecd-1ff3b4c4e004 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:58.266951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76994001-a206-43ff-9ecd-1ff3b4c4e004 service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:58.267119 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76994001-a206-43ff-9ecd-1ff3b4c4e004 service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:58.267258 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76994001-a206-43ff-9ecd-1ff3b4c4e004 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:58.360719 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48282db2-deb5-4840-974c-6c859836f0f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:58.362303 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48282db2-deb5-4840-974c-6c859836f0f6 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:58.373788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 909/1811] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:58.513683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76994001-a206-43ff-9ecd-1ff3b4c4e004 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:58.516771 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76994001-a206-43ff-9ecd-1ff3b4c4e004 service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:58.540820 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48282db2-deb5-4840-974c-6c859836f0f6 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:58.543126 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-48282db2-deb5-4840-974c-6c859836f0f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:58.547330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 903/1812] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:44:58.712661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48282db2-deb5-4840-974c-6c859836f0f6 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:58.722762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d7867-c49c-4b90-a8bd-07d189e53741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:58.723706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d7867-c49c-4b90-a8bd-07d189e53741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:58.724173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d7867-c49c-4b90-a8bd-07d189e53741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:58.724694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d7867-c49c-4b90-a8bd-07d189e53741 admin admin] PATH_INFO: `/v3/projects/4c95826184d84a2f9e01cbc43c0e7ee9` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:58.727006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d7867-c49c-4b90-a8bd-07d189e53741 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c95826184d84a2f9e01cbc43c0e7ee9)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:58.731184 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d7867-c49c-4b90-a8bd-07d189e53741 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:58.771464 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76994001-a206-43ff-9ecd-1ff3b4c4e004 service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:58.792587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76994001-a206-43ff-9ecd-1ff3b4c4e004 service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:58.797623 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafba2ec-23f0-4dbb-bd9f-025d8685e4b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:58.798302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafba2ec-23f0-4dbb-bd9f-025d8685e4b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:58.798619 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafba2ec-23f0-4dbb-bd9f-025d8685e4b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:58.799118 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafba2ec-23f0-4dbb-bd9f-025d8685e4b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:58.802121 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-dafba2ec-23f0-4dbb-bd9f-025d8685e4b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:58.837127 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-3b3d7867-c49c-4b90-a8bd-07d189e53741 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c95826184d84a2f9e01cbc43c0e7ee9', 'request_id': 'req-3b3d7867-c49c-4b90-a8bd-07d189e53741'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.851284 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-3b3d7867-c49c-4b90-a8bd-07d189e53741 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c95826184d84a2f9e01cbc43c0e7ee9', 'request_id': 'req-3b3d7867-c49c-4b90-a8bd-07d189e53741'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.866381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-3b3d7867-c49c-4b90-a8bd-07d189e53741 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c95826184d84a2f9e01cbc43c0e7ee9', 'request_id': 'req-3b3d7867-c49c-4b90-a8bd-07d189e53741'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.886928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-3b3d7867-c49c-4b90-a8bd-07d189e53741 admin admin] The token cache is being invalidate because project 4c95826184d84a2f9e01cbc43c0e7ee9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:58.887552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-3b3d7867-c49c-4b90-a8bd-07d189e53741 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.889318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-3b3d7867-c49c-4b90-a8bd-07d189e53741 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.892351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 910/1813] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:58 2022] DELETE /identity/v3/projects/4c95826184d84a2f9e01cbc43c0e7ee9 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:44:58.900244 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444d4a9-7b34-4143-8571-ab41fe98de3f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:58.900718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444d4a9-7b34-4143-8571-ab41fe98de3f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:58.901177 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444d4a9-7b34-4143-8571-ab41fe98de3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:58.921901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafba2ec-23f0-4dbb-bd9f-025d8685e4b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:58.924829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafba2ec-23f0-4dbb-bd9f-025d8685e4b0 service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:58.936241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-5444d4a9-7b34-4143-8571-ab41fe98de3f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:58.938953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5444d4a9-7b34-4143-8571-ab41fe98de3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:58.940298 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 904/1814] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:44:59.027363 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:59.113444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 911/1815] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:59.124663 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987cb8b-37b2-4923-95db-54c1641bf95f None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:59.124935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987cb8b-37b2-4923-95db-54c1641bf95f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:59.125222 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987cb8b-37b2-4923-95db-54c1641bf95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:59.165113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-f987cb8b-37b2-4923-95db-54c1641bf95f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:59.173286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f987cb8b-37b2-4923-95db-54c1641bf95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:59.219377 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 905/1816] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:59.226806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444d4a9-7b34-4143-8571-ab41fe98de3f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:59.268091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444d4a9-7b34-4143-8571-ab41fe98de3f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:59.287899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5f3bd-fb4b-46b6-ba4c-a3d72fccf233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:59.288542 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5f3bd-fb4b-46b6-ba4c-a3d72fccf233 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:59.288830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5f3bd-fb4b-46b6-ba4c-a3d72fccf233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:59.289104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5f3bd-fb4b-46b6-ba4c-a3d72fccf233 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:59.289775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5f3bd-fb4b-46b6-ba4c-a3d72fccf233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:44:59.290079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5f3bd-fb4b-46b6-ba4c-a3d72fccf233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:59.291750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5f3bd-fb4b-46b6-ba4c-a3d72fccf233 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:59.313452 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 912/1817] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:44:59.320643 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987cb8b-37b2-4923-95db-54c1641bf95f None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:59.336921 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987cb8b-37b2-4923-95db-54c1641bf95f None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:59.348477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05527e-4291-4f71-9c72-1758e91e61e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:59.349799 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05527e-4291-4f71-9c72-1758e91e61e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:59.350254 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05527e-4291-4f71-9c72-1758e91e61e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:59.350704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05527e-4291-4f71-9c72-1758e91e61e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:59.351857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05527e-4291-4f71-9c72-1758e91e61e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:59.355768 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05527e-4291-4f71-9c72-1758e91e61e2 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:59.451598 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 906/1818] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:59 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:59.456587 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5f3bd-fb4b-46b6-ba4c-a3d72fccf233 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:59.469399 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5f3bd-fb4b-46b6-ba4c-a3d72fccf233 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:59.571156 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09a968-2e2b-4833-a89b-23410198e78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:59.572151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09a968-2e2b-4833-a89b-23410198e78a admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:59.572669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09a968-2e2b-4833-a89b-23410198e78a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:59.573156 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09a968-2e2b-4833-a89b-23410198e78a admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:59.574025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09a968-2e2b-4833-a89b-23410198e78a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:59.577413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09a968-2e2b-4833-a89b-23410198e78a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:59.627574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 913/1819] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:44:59 2022] POST /identity/v3/users => generated 284 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:44:59.636721 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05527e-4291-4f71-9c72-1758e91e61e2 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:59.658825 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05527e-4291-4f71-9c72-1758e91e61e2 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:59.667531 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba71dc-e903-4a79-b56d-4daac6c4fa68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:59.669430 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba71dc-e903-4a79-b56d-4daac6c4fa68 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:59.669430 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba71dc-e903-4a79-b56d-4daac6c4fa68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:59.669854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba71dc-e903-4a79-b56d-4daac6c4fa68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:59.671228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba71dc-e903-4a79-b56d-4daac6c4fa68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:59.674200 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba71dc-e903-4a79-b56d-4daac6c4fa68 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:59.695474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 907/1820] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:59.701704 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09a968-2e2b-4833-a89b-23410198e78a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:59.721077 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09a968-2e2b-4833-a89b-23410198e78a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:59.728995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687a85a-9f59-4e8e-b094-b73c00e312a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:59.730308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687a85a-9f59-4e8e-b094-b73c00e312a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:59.730573 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687a85a-9f59-4e8e-b094-b73c00e312a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:59.731040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687a85a-9f59-4e8e-b094-b73c00e312a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:59.732939 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687a85a-9f59-4e8e-b094-b73c00e312a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:59.736683 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687a85a-9f59-4e8e-b094-b73c00e312a3 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:59.755316 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 914/1821] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:59.761323 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba71dc-e903-4a79-b56d-4daac6c4fa68 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:59.783459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba71dc-e903-4a79-b56d-4daac6c4fa68 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:59.790959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90922905-eb60-4a71-9bff-7abbfc5aa7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:59.792580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90922905-eb60-4a71-9bff-7abbfc5aa7e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:59.793019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90922905-eb60-4a71-9bff-7abbfc5aa7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:59.793572 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90922905-eb60-4a71-9bff-7abbfc5aa7e8 admin admin] PATH_INFO: `/v3/projects/6476c04596df43f4ba7c0af42ff340f2/users/9b278c41191541e7bf182903fb9d2c14/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:59.813663 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:59.826065 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90922905-eb60-4a71-9bff-7abbfc5aa7e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6476c04596df43f4ba7c0af42ff340f2, user_id=9b278c41191541e7bf182903fb9d2c14, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:59.832522 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90922905-eb60-4a71-9bff-7abbfc5aa7e8 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:59.861548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 908/1822] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:59 2022] PUT /identity/v3/projects/6476c04596df43f4ba7c0af42ff340f2/users/9b278c41191541e7bf182903fb9d2c14/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:59.878294 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c3217-0c9a-4bf0-af05-9c3edd569259 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:59.878731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c3217-0c9a-4bf0-af05-9c3edd569259 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:59.879147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c3217-0c9a-4bf0-af05-9c3edd569259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:59.927475 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1f9c3217-0c9a-4bf0-af05-9c3edd569259 None None] MFA Rules not processed for user `9b278c41191541e7bf182903fb9d2c14`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:59.929591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1f9c3217-0c9a-4bf0-af05-9c3edd569259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:00.036580 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 915/1823] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:44:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:00.041640 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90922905-eb60-4a71-9bff-7abbfc5aa7e8 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:00.115480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90922905-eb60-4a71-9bff-7abbfc5aa7e8 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:00.121293 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c544e-a0ae-49db-83e0-7e10935c4a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:00.122073 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c544e-a0ae-49db-83e0-7e10935c4a8c admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:00.122463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c544e-a0ae-49db-83e0-7e10935c4a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:00.122803 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c544e-a0ae-49db-83e0-7e10935c4a8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:00.123372 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c544e-a0ae-49db-83e0-7e10935c4a8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:00.125591 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c544e-a0ae-49db-83e0-7e10935c4a8c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:00.159165 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 909/1824] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:00 2022] POST /identity/v3/projects => generated 369 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:00.164375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c3217-0c9a-4bf0-af05-9c3edd569259 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:00.177346 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c3217-0c9a-4bf0-af05-9c3edd569259 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:00.221693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:00.262484 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63177f5-d780-457e-8bd9-156f3939f9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:00.263378 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63177f5-d780-457e-8bd9-156f3939f9f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:00.263857 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63177f5-d780-457e-8bd9-156f3939f9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:00.264340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63177f5-d780-457e-8bd9-156f3939f9f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:00.265785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63177f5-d780-457e-8bd9-156f3939f9f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:00.268544 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63177f5-d780-457e-8bd9-156f3939f9f7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:00.305093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 916/1825] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:00 2022] POST /identity/v3/users => generated 291 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:45:00.310107 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c544e-a0ae-49db-83e0-7e10935c4a8c admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:00.329325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c544e-a0ae-49db-83e0-7e10935c4a8c admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:00.337282 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa7011-3f51-45cc-9311-64e027d10eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:00.338243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa7011-3f51-45cc-9311-64e027d10eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:00.338724 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa7011-3f51-45cc-9311-64e027d10eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:00.339178 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa7011-3f51-45cc-9311-64e027d10eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:00.340358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa7011-3f51-45cc-9311-64e027d10eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:00.343740 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa7011-3f51-45cc-9311-64e027d10eb9 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:00.365358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 910/1826] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:00.373328 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63177f5-d780-457e-8bd9-156f3939f9f7 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:00.388187 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63177f5-d780-457e-8bd9-156f3939f9f7 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:00.395838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db229463-b0b0-489f-a39a-4b473f246fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:00.396765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db229463-b0b0-489f-a39a-4b473f246fda admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:00.397210 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db229463-b0b0-489f-a39a-4b473f246fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:00.397704 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db229463-b0b0-489f-a39a-4b473f246fda admin admin] PATH_INFO: `/v3/projects/8428008d569e4a0da2201825fdb9b8d4/users/62c7d1cd4b7e4677a261f41ae315f399/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:00.418948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db229463-b0b0-489f-a39a-4b473f246fda admin admin] RBAC: Authorizing `identity:create_grant(project_id=8428008d569e4a0da2201825fdb9b8d4, user_id=62c7d1cd4b7e4677a261f41ae315f399, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:00.422257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db229463-b0b0-489f-a39a-4b473f246fda admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:00.451074 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 917/1827] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:00 2022] PUT /identity/v3/projects/8428008d569e4a0da2201825fdb9b8d4/users/62c7d1cd4b7e4677a261f41ae315f399/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:00.452955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa7011-3f51-45cc-9311-64e027d10eb9 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:00.475361 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa7011-3f51-45cc-9311-64e027d10eb9 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:00.620162 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31936949-51e8-431e-bde7-658ae79e79e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:00.621711 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31936949-51e8-431e-bde7-658ae79e79e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:00.622418 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31936949-51e8-431e-bde7-658ae79e79e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:00.623504 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31936949-51e8-431e-bde7-658ae79e79e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:00.636616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31936949-51e8-431e-bde7-658ae79e79e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:00.643420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31936949-51e8-431e-bde7-658ae79e79e2 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:00.675672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 911/1828] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:45:00.682380 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db229463-b0b0-489f-a39a-4b473f246fda admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:00.705482 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db229463-b0b0-489f-a39a-4b473f246fda admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:00.714211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99857c59-afa3-4af4-94fe-532363d6f77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:00.715861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99857c59-afa3-4af4-94fe-532363d6f77a admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:00.716393 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99857c59-afa3-4af4-94fe-532363d6f77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:00.716927 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99857c59-afa3-4af4-94fe-532363d6f77a admin admin] PATH_INFO: `/v3/projects/8428008d569e4a0da2201825fdb9b8d4/users/62c7d1cd4b7e4677a261f41ae315f399/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:00.720769 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99857c59-afa3-4af4-94fe-532363d6f77a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8428008d569e4a0da2201825fdb9b8d4, user_id=62c7d1cd4b7e4677a261f41ae315f399, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:00.726901 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99857c59-afa3-4af4-94fe-532363d6f77a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:00.749947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 918/1829] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:00 2022] PUT /identity/v3/projects/8428008d569e4a0da2201825fdb9b8d4/users/62c7d1cd4b7e4677a261f41ae315f399/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:00.766846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13404ea-b3c0-406d-8455-6838072c84ae None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:00.767306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13404ea-b3c0-406d-8455-6838072c84ae None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:00.767818 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13404ea-b3c0-406d-8455-6838072c84ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:00.830011 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-f13404ea-b3c0-406d-8455-6838072c84ae None None] MFA Rules not processed for user `62c7d1cd4b7e4677a261f41ae315f399`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:00.830011 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f13404ea-b3c0-406d-8455-6838072c84ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:00.952453 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:01.021335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 912/1830] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:01.076154 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99857c59-afa3-4af4-94fe-532363d6f77a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:01.079349 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-99857c59-afa3-4af4-94fe-532363d6f77a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:01.233749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99857c59-afa3-4af4-94fe-532363d6f77a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:01.239224 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd0351-99b0-4fc4-b42d-e5f557221e75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:01.240122 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd0351-99b0-4fc4-b42d-e5f557221e75 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:01.240569 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd0351-99b0-4fc4-b42d-e5f557221e75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:01.240980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd0351-99b0-4fc4-b42d-e5f557221e75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:01.257744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd0351-99b0-4fc4-b42d-e5f557221e75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:01.260057 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd0351-99b0-4fc4-b42d-e5f557221e75 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:01.276872 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 919/1831] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:01.367667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13404ea-b3c0-406d-8455-6838072c84ae None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:01.369790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f13404ea-b3c0-406d-8455-6838072c84ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:01.387641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13404ea-b3c0-406d-8455-6838072c84ae None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:01.392846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f91fc5-5b6a-41df-aaca-9bec6fd8311a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:01.393481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f91fc5-5b6a-41df-aaca-9bec6fd8311a service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:01.393946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f91fc5-5b6a-41df-aaca-9bec6fd8311a service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:01.394237 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f91fc5-5b6a-41df-aaca-9bec6fd8311a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:01.499074 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f91fc5-5b6a-41df-aaca-9bec6fd8311a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:01.501326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f91fc5-5b6a-41df-aaca-9bec6fd8311a service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:01.518849 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 913/1832] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:01.635252 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd0351-99b0-4fc4-b42d-e5f557221e75 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:01.637868 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-facd0351-99b0-4fc4-b42d-e5f557221e75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:01.852608 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd0351-99b0-4fc4-b42d-e5f557221e75 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:01.861688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b156e-8bc9-4520-8a74-5902017fb9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:01.863074 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b156e-8bc9-4520-8a74-5902017fb9ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:01.863676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b156e-8bc9-4520-8a74-5902017fb9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:01.864384 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b156e-8bc9-4520-8a74-5902017fb9ee admin admin] PATH_INFO: `/v3/users/1df8719693204e4f8737478cba11dc31` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:01.866711 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b156e-8bc9-4520-8a74-5902017fb9ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=1df8719693204e4f8737478cba11dc31)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:01.870620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b156e-8bc9-4520-8a74-5902017fb9ee admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:01.961095 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:01.986875 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e68b156e-8bc9-4520-8a74-5902017fb9ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1df8719693204e4f8737478cba11dc31'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:02.002440 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e68b156e-8bc9-4520-8a74-5902017fb9ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1df8719693204e4f8737478cba11dc31'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:02.026586 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e68b156e-8bc9-4520-8a74-5902017fb9ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1df8719693204e4f8737478cba11dc31'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:02.027664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e68b156e-8bc9-4520-8a74-5902017fb9ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1df8719693204e4f8737478cba11dc31'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:02.057615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e68b156e-8bc9-4520-8a74-5902017fb9ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1df8719693204e4f8737478cba11dc31'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:02.070017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 920/1833] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:01 2022] DELETE /identity/v3/users/1df8719693204e4f8737478cba11dc31 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:02.079219 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657d988-2285-427f-a087-92733525a99e None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:02.079561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657d988-2285-427f-a087-92733525a99e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:02.080708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657d988-2285-427f-a087-92733525a99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:02.121742 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-b657d988-2285-427f-a087-92733525a99e None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:02.123619 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b657d988-2285-427f-a087-92733525a99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:02.252234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b156e-8bc9-4520-8a74-5902017fb9ee admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:02.254557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e68b156e-8bc9-4520-8a74-5902017fb9ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:02.359800 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 914/1834] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:02.405236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657d988-2285-427f-a087-92733525a99e None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:02.409680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b657d988-2285-427f-a087-92733525a99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:02.475648 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b156e-8bc9-4520-8a74-5902017fb9ee admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:02.512707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00347c5-9f7f-4d61-bb33-fcc8590c98e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:02.515121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00347c5-9f7f-4d61-bb33-fcc8590c98e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:02.518908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00347c5-9f7f-4d61-bb33-fcc8590c98e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:02.519608 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00347c5-9f7f-4d61-bb33-fcc8590c98e7 admin admin] PATH_INFO: `/v3/users/6068bc03b9014132bf7a9b85c9f48c58` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:02.532795 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00347c5-9f7f-4d61-bb33-fcc8590c98e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6068bc03b9014132bf7a9b85c9f48c58)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:02.537020 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00347c5-9f7f-4d61-bb33-fcc8590c98e7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:02.635004 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657d988-2285-427f-a087-92733525a99e None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:02.658194 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220048d1-11df-41a9-9160-b746a54e222b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:02.659211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220048d1-11df-41a9-9160-b746a54e222b service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:02.659664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220048d1-11df-41a9-9160-b746a54e222b service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:02.660184 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220048d1-11df-41a9-9160-b746a54e222b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:02.682572 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220048d1-11df-41a9-9160-b746a54e222b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:02.684476 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220048d1-11df-41a9-9160-b746a54e222b service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:02.699028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c00347c5-9f7f-4d61-bb33-fcc8590c98e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6068bc03b9014132bf7a9b85c9f48c58'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:02.699382 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 915/1835] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:02.708677 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c00347c5-9f7f-4d61-bb33-fcc8590c98e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6068bc03b9014132bf7a9b85c9f48c58'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:02.721403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c00347c5-9f7f-4d61-bb33-fcc8590c98e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6068bc03b9014132bf7a9b85c9f48c58'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:02.721746 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c00347c5-9f7f-4d61-bb33-fcc8590c98e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6068bc03b9014132bf7a9b85c9f48c58'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:02.755086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c00347c5-9f7f-4d61-bb33-fcc8590c98e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6068bc03b9014132bf7a9b85c9f48c58'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:02.768033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 921/1836] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:02 2022] DELETE /identity/v3/users/6068bc03b9014132bf7a9b85c9f48c58 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:02.776298 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126033a-2583-452b-95ef-e410e756f5b4 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:02.776792 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126033a-2583-452b-95ef-e410e756f5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:02.777458 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126033a-2583-452b-95ef-e410e756f5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:02.834829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-4126033a-2583-452b-95ef-e410e756f5b4 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:02.837455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4126033a-2583-452b-95ef-e410e756f5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:02.916039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:03.000731 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 916/1837] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:03.037064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00347c5-9f7f-4d61-bb33-fcc8590c98e7 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:03.039166 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c00347c5-9f7f-4d61-bb33-fcc8590c98e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:03.126537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:03.188568 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00347c5-9f7f-4d61-bb33-fcc8590c98e7 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:03.196650 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa12e7b-da59-4215-84a9-27dd8d66e14f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:03.198008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa12e7b-da59-4215-84a9-27dd8d66e14f service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:03.198472 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa12e7b-da59-4215-84a9-27dd8d66e14f service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:03.198998 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa12e7b-da59-4215-84a9-27dd8d66e14f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:03.221726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa12e7b-da59-4215-84a9-27dd8d66e14f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:03.224905 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa12e7b-da59-4215-84a9-27dd8d66e14f service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:03.258166 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 922/1838] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:03.355023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126033a-2583-452b-95ef-e410e756f5b4 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:03.358985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4126033a-2583-452b-95ef-e410e756f5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:03.391250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126033a-2583-452b-95ef-e410e756f5b4 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:03.406901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b7d5a-88d4-4cef-83af-568eb012a818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:03.409431 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b7d5a-88d4-4cef-83af-568eb012a818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:03.410103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b7d5a-88d4-4cef-83af-568eb012a818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:03.410943 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b7d5a-88d4-4cef-83af-568eb012a818 admin admin] PATH_INFO: `/v3/projects/a49024fae001434a996e45222afbb603` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:03.415248 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b7d5a-88d4-4cef-83af-568eb012a818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a49024fae001434a996e45222afbb603)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:03.418561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b7d5a-88d4-4cef-83af-568eb012a818 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:03.501261 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-be5b7d5a-88d4-4cef-83af-568eb012a818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a49024fae001434a996e45222afbb603', 'request_id': 'req-be5b7d5a-88d4-4cef-83af-568eb012a818'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:03.520181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-be5b7d5a-88d4-4cef-83af-568eb012a818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a49024fae001434a996e45222afbb603', 'request_id': 'req-be5b7d5a-88d4-4cef-83af-568eb012a818'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:03.536732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-be5b7d5a-88d4-4cef-83af-568eb012a818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a49024fae001434a996e45222afbb603', 'request_id': 'req-be5b7d5a-88d4-4cef-83af-568eb012a818'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:03.559040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-be5b7d5a-88d4-4cef-83af-568eb012a818 admin admin] The token cache is being invalidate because project a49024fae001434a996e45222afbb603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:03.559446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-be5b7d5a-88d4-4cef-83af-568eb012a818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:03.560805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-be5b7d5a-88d4-4cef-83af-568eb012a818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:03.564063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 917/1839] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:03 2022] DELETE /identity/v3/projects/a49024fae001434a996e45222afbb603 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:03.568591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa12e7b-da59-4215-84a9-27dd8d66e14f service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:03.599265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa12e7b-da59-4215-84a9-27dd8d66e14f service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:03.823196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422391c-ad0b-4337-ae2c-66941bd74c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:03.825164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422391c-ad0b-4337-ae2c-66941bd74c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:03.825164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422391c-ad0b-4337-ae2c-66941bd74c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:03.825451 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422391c-ad0b-4337-ae2c-66941bd74c3c admin admin] PATH_INFO: `/v3/users/2ba9fc062a1745eabc289b5586517b48` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:03.829149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422391c-ad0b-4337-ae2c-66941bd74c3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ba9fc062a1745eabc289b5586517b48)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:03.833924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422391c-ad0b-4337-ae2c-66941bd74c3c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:03.972512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1422391c-ad0b-4337-ae2c-66941bd74c3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ba9fc062a1745eabc289b5586517b48'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:04.010160 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1422391c-ad0b-4337-ae2c-66941bd74c3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ba9fc062a1745eabc289b5586517b48'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:04.038862 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1422391c-ad0b-4337-ae2c-66941bd74c3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ba9fc062a1745eabc289b5586517b48'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:04.039853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1422391c-ad0b-4337-ae2c-66941bd74c3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ba9fc062a1745eabc289b5586517b48'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:04.078549 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1422391c-ad0b-4337-ae2c-66941bd74c3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ba9fc062a1745eabc289b5586517b48'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:04.111216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 923/1840] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:03 2022] DELETE /identity/v3/users/2ba9fc062a1745eabc289b5586517b48 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:04.354740 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0fab9-e220-4abc-87b0-45db21b39fec None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:04.355245 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0fab9-e220-4abc-87b0-45db21b39fec None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:04.355852 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0fab9-e220-4abc-87b0-45db21b39fec None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:04.358522 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 918/1841] 10.222.0.25 () {54 vars in 921 bytes} [Tue Aug 30 14:45:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:45:04.380593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903aa8fa-a1d6-4cb0-b7e2-8ddebd987909 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:04.383001 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903aa8fa-a1d6-4cb0-b7e2-8ddebd987909 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:04.383758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903aa8fa-a1d6-4cb0-b7e2-8ddebd987909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:04.432889 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0fab9-e220-4abc-87b0-45db21b39fec None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:04.442947 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a8f0fab9-e220-4abc-87b0-45db21b39fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:04.453380 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-903aa8fa-a1d6-4cb0-b7e2-8ddebd987909 None None] MFA Rules not processed for user `cc95bbdafb0c4f34ad0662bc2e3ba4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:04.455339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-903aa8fa-a1d6-4cb0-b7e2-8ddebd987909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:04.611692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:04.656775 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0fab9-e220-4abc-87b0-45db21b39fec None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:04.662364 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c7028-81a4-4351-8d7d-a4f92ebf77cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:04.663002 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c7028-81a4-4351-8d7d-a4f92ebf77cd service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:04.663283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c7028-81a4-4351-8d7d-a4f92ebf77cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:04.663552 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c7028-81a4-4351-8d7d-a4f92ebf77cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:04.681903 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5c1c7028-81a4-4351-8d7d-a4f92ebf77cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:04.716922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 924/1842] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Aug 30 14:45:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:04.884697 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c7028-81a4-4351-8d7d-a4f92ebf77cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:04.893415 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c7028-81a4-4351-8d7d-a4f92ebf77cd service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:04.907035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:04.920289 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 919/1843] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:05.106436 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903aa8fa-a1d6-4cb0-b7e2-8ddebd987909 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:05.109025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-903aa8fa-a1d6-4cb0-b7e2-8ddebd987909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:05.138521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c7028-81a4-4351-8d7d-a4f92ebf77cd service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:05.140308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5c1c7028-81a4-4351-8d7d-a4f92ebf77cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:05.153636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903aa8fa-a1d6-4cb0-b7e2-8ddebd987909 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:05.175322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117e881-6128-48e5-8bef-f33d84dc7e3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:05.176049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117e881-6128-48e5-8bef-f33d84dc7e3b service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:05.176322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117e881-6128-48e5-8bef-f33d84dc7e3b service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:05.176662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117e881-6128-48e5-8bef-f33d84dc7e3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:05.218709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117e881-6128-48e5-8bef-f33d84dc7e3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:05.220529 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117e881-6128-48e5-8bef-f33d84dc7e3b service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:05.245320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 925/1844] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:05.246440 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117e881-6128-48e5-8bef-f33d84dc7e3b service nova] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:05.250808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6117e881-6128-48e5-8bef-f33d84dc7e3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:05.325425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c7028-81a4-4351-8d7d-a4f92ebf77cd service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:05.334464 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:05.335592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:05.336036 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:05.336482 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da admin admin] PATH_INFO: `/v3/projects/902ad629e7e54ba0bef88ca561dcb819` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:05.339601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da admin admin] RBAC: Authorizing `identity:delete_project(project_id=902ad629e7e54ba0bef88ca561dcb819)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:05.343820 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:05.418587 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117e881-6128-48e5-8bef-f33d84dc7e3b service nova] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:05.427319 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae11a2-0bef-4d4d-9b03-8604f8cba4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:05.428936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae11a2-0bef-4d4d-9b03-8604f8cba4b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:05.429653 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae11a2-0bef-4d4d-9b03-8604f8cba4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:05.430306 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae11a2-0bef-4d4d-9b03-8604f8cba4b8 admin admin] PATH_INFO: `/v3/projects/144a5ee24aa44d4392da313cd405c715` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:05.433494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae11a2-0bef-4d4d-9b03-8604f8cba4b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=144a5ee24aa44d4392da313cd405c715)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:05.439293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae11a2-0bef-4d4d-9b03-8604f8cba4b8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:05.456973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '902ad629e7e54ba0bef88ca561dcb819', 'request_id': 'req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:05.468171 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '902ad629e7e54ba0bef88ca561dcb819', 'request_id': 'req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:05.478779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '902ad629e7e54ba0bef88ca561dcb819', 'request_id': 'req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:05.501489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da admin admin] The token cache is being invalidate because project 902ad629e7e54ba0bef88ca561dcb819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:05.501804 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:05.502578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:05.504350 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 920/1845] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:05 2022] DELETE /identity/v3/projects/902ad629e7e54ba0bef88ca561dcb819 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:45:05.508689 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:05.510213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:05.569413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1eae11a2-0bef-4d4d-9b03-8604f8cba4b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '144a5ee24aa44d4392da313cd405c715', 'request_id': 'req-1eae11a2-0bef-4d4d-9b03-8604f8cba4b8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:05.593972 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1eae11a2-0bef-4d4d-9b03-8604f8cba4b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '144a5ee24aa44d4392da313cd405c715', 'request_id': 'req-1eae11a2-0bef-4d4d-9b03-8604f8cba4b8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:05.603649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1eae11a2-0bef-4d4d-9b03-8604f8cba4b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '144a5ee24aa44d4392da313cd405c715', 'request_id': 'req-1eae11a2-0bef-4d4d-9b03-8604f8cba4b8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:05.626086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1eae11a2-0bef-4d4d-9b03-8604f8cba4b8 admin admin] The token cache is being invalidate because project 144a5ee24aa44d4392da313cd405c715 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:05.626608 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1eae11a2-0bef-4d4d-9b03-8604f8cba4b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:05.628289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1eae11a2-0bef-4d4d-9b03-8604f8cba4b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:05.632823 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 926/1846] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:05 2022] DELETE /identity/v3/projects/144a5ee24aa44d4392da313cd405c715 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:05.638609 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdeda6c-47d8-42f3-b07e-f0856f9eab60 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:05.638918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdeda6c-47d8-42f3-b07e-f0856f9eab60 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:05.639174 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdeda6c-47d8-42f3-b07e-f0856f9eab60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:05.674526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a08a5-1384-4fa8-bdf2-c6e18fa6f1da admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:05.678251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1fdeda6c-47d8-42f3-b07e-f0856f9eab60 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:05.679701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1fdeda6c-47d8-42f3-b07e-f0856f9eab60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:05.784402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 927/1847] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:05.784588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:05.784730 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ec082-d362-4b4c-a7a1-672115669ae6 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:05.784730 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ec082-d362-4b4c-a7a1-672115669ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:05.784730 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ec082-d362-4b4c-a7a1-672115669ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:05.805315 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:05.820816 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8370ee1-8647-4968-9960-32b8e279fcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:05.821961 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8370ee1-8647-4968-9960-32b8e279fcfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:05.823129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8370ee1-8647-4968-9960-32b8e279fcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:05.823129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8370ee1-8647-4968-9960-32b8e279fcfa admin admin] PATH_INFO: `/v3/users/809a0d8eb0eb4efda036129a65cc729a` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:05.825243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8370ee1-8647-4968-9960-32b8e279fcfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=809a0d8eb0eb4efda036129a65cc729a)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:05.829548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8370ee1-8647-4968-9960-32b8e279fcfa admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:05.857403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d49ec082-d362-4b4c-a7a1-672115669ae6 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:05.859556 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d49ec082-d362-4b4c-a7a1-672115669ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:05.890085 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 928/1848] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:05.895333 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ec082-d362-4b4c-a7a1-672115669ae6 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:05.924528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ec082-d362-4b4c-a7a1-672115669ae6 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:05.937357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ce127-61b3-4584-b8ce-2f35490829e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:05.938009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ce127-61b3-4584-b8ce-2f35490829e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:05.938286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ce127-61b3-4584-b8ce-2f35490829e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:05.938561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ce127-61b3-4584-b8ce-2f35490829e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:05.939246 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ce127-61b3-4584-b8ce-2f35490829e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:45:05.939547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ce127-61b3-4584-b8ce-2f35490829e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:05.942045 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ce127-61b3-4584-b8ce-2f35490829e1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:05.961419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 929/1849] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:45:05.968051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ce127-61b3-4584-b8ce-2f35490829e1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:05.968919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a8370ee1-8647-4968-9960-32b8e279fcfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '809a0d8eb0eb4efda036129a65cc729a'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:05.970198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a8370ee1-8647-4968-9960-32b8e279fcfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '809a0d8eb0eb4efda036129a65cc729a'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:05.971637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a8370ee1-8647-4968-9960-32b8e279fcfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '809a0d8eb0eb4efda036129a65cc729a'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:05.980219 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ce127-61b3-4584-b8ce-2f35490829e1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:05.986973 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6efb7c-febd-4943-95ea-49690daf1c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:05.987616 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6efb7c-febd-4943-95ea-49690daf1c9a admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:05.987897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6efb7c-febd-4943-95ea-49690daf1c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:05.988250 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6efb7c-febd-4943-95ea-49690daf1c9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:05.988937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6efb7c-febd-4943-95ea-49690daf1c9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:06.000114 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6efb7c-febd-4943-95ea-49690daf1c9a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:06.003846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a8370ee1-8647-4968-9960-32b8e279fcfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '809a0d8eb0eb4efda036129a65cc729a'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:06.025381 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a8370ee1-8647-4968-9960-32b8e279fcfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '809a0d8eb0eb4efda036129a65cc729a'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:06.044216 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 921/1850] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:05 2022] DELETE /identity/v3/users/809a0d8eb0eb4efda036129a65cc729a => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:45:06.063441 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 930/1851] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:05 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:45:06.068806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8370ee1-8647-4968-9960-32b8e279fcfa admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:06.071424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a8370ee1-8647-4968-9960-32b8e279fcfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:06.180055 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8370ee1-8647-4968-9960-32b8e279fcfa admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:06.281882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbc2b2-c541-4005-90b6-48c3e4419ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:06.283755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbc2b2-c541-4005-90b6-48c3e4419ade admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:06.284137 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbc2b2-c541-4005-90b6-48c3e4419ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:06.284517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbc2b2-c541-4005-90b6-48c3e4419ade admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:06.285332 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbc2b2-c541-4005-90b6-48c3e4419ade admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:06.287709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbc2b2-c541-4005-90b6-48c3e4419ade admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:06.306154 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:06.333898 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 922/1852] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:06 2022] POST /identity/v3/users => generated 293 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:06.341228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6efb7c-febd-4943-95ea-49690daf1c9a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:06.360936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6efb7c-febd-4943-95ea-49690daf1c9a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:06.370180 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10fae0-a76f-43d9-9fe8-9f6276b325e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:06.370982 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10fae0-a76f-43d9-9fe8-9f6276b325e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:06.371380 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10fae0-a76f-43d9-9fe8-9f6276b325e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:06.371865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10fae0-a76f-43d9-9fe8-9f6276b325e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:06.373706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10fae0-a76f-43d9-9fe8-9f6276b325e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:06.376956 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10fae0-a76f-43d9-9fe8-9f6276b325e2 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:06.397459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 931/1853] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:06.404597 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbc2b2-c541-4005-90b6-48c3e4419ade admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:06.422682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbc2b2-c541-4005-90b6-48c3e4419ade admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:06.442106 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac44b4-5eea-4530-89d7-e16be6579a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:06.443872 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac44b4-5eea-4530-89d7-e16be6579a31 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:06.444858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac44b4-5eea-4530-89d7-e16be6579a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:06.445578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac44b4-5eea-4530-89d7-e16be6579a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:06.447942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac44b4-5eea-4530-89d7-e16be6579a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:06.451772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac44b4-5eea-4530-89d7-e16be6579a31 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:06.474382 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 923/1854] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:06.478561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10fae0-a76f-43d9-9fe8-9f6276b325e2 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:06.495154 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10fae0-a76f-43d9-9fe8-9f6276b325e2 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:06.504243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744c9b0-63d4-44db-94a2-fbcfd4f6be00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:06.506877 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744c9b0-63d4-44db-94a2-fbcfd4f6be00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:06.507714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744c9b0-63d4-44db-94a2-fbcfd4f6be00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:06.508194 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744c9b0-63d4-44db-94a2-fbcfd4f6be00 admin admin] PATH_INFO: `/v3/projects/747547b986f1448ca0e9e43a76906a0f/users/c7250085a1f940ab87a26c5ced51ca33/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:06.528499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744c9b0-63d4-44db-94a2-fbcfd4f6be00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=747547b986f1448ca0e9e43a76906a0f, user_id=c7250085a1f940ab87a26c5ced51ca33, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:06.532545 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744c9b0-63d4-44db-94a2-fbcfd4f6be00 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:06.554733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 932/1855] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:06 2022] PUT /identity/v3/projects/747547b986f1448ca0e9e43a76906a0f/users/c7250085a1f940ab87a26c5ced51ca33/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:45:06.567505 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32694f62-bd1c-4581-ae2c-e0749d63233a None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:06.567650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32694f62-bd1c-4581-ae2c-e0749d63233a None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:06.568330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32694f62-bd1c-4581-ae2c-e0749d63233a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:06.684218 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-32694f62-bd1c-4581-ae2c-e0749d63233a None None] MFA Rules not processed for user `c7250085a1f940ab87a26c5ced51ca33`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:06.688736 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-32694f62-bd1c-4581-ae2c-e0749d63233a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:06.769141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744c9b0-63d4-44db-94a2-fbcfd4f6be00 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:06.772166 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6744c9b0-63d4-44db-94a2-fbcfd4f6be00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:06.824288 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 924/1856] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:06.830396 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32694f62-bd1c-4581-ae2c-e0749d63233a None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:06.930909 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744c9b0-63d4-44db-94a2-fbcfd4f6be00 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:06.937231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2949a-c125-4836-bafd-4633f0eb215e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:06.938493 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2949a-c125-4836-bafd-4633f0eb215e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:06.938671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2949a-c125-4836-bafd-4633f0eb215e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:06.938948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2949a-c125-4836-bafd-4633f0eb215e admin admin] PATH_INFO: `/v3/users/9b278c41191541e7bf182903fb9d2c14` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:06.940474 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2949a-c125-4836-bafd-4633f0eb215e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b278c41191541e7bf182903fb9d2c14)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:06.944538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2949a-c125-4836-bafd-4633f0eb215e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:07.000809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32694f62-bd1c-4581-ae2c-e0749d63233a None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:07.009681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171fab7-992d-4b5e-a753-8dfc89b6497c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:07.010959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171fab7-992d-4b5e-a753-8dfc89b6497c admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:07.011531 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171fab7-992d-4b5e-a753-8dfc89b6497c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:07.012202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171fab7-992d-4b5e-a753-8dfc89b6497c admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:07.013335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171fab7-992d-4b5e-a753-8dfc89b6497c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:07.017312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171fab7-992d-4b5e-a753-8dfc89b6497c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:07.023789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1aa2949a-c125-4836-bafd-4633f0eb215e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b278c41191541e7bf182903fb9d2c14'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:07.045803 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1aa2949a-c125-4836-bafd-4633f0eb215e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b278c41191541e7bf182903fb9d2c14'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:07.059743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1aa2949a-c125-4836-bafd-4633f0eb215e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b278c41191541e7bf182903fb9d2c14'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:07.065903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1aa2949a-c125-4836-bafd-4633f0eb215e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b278c41191541e7bf182903fb9d2c14'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:07.080360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 925/1857] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:06 2022] POST /identity/v3/projects => generated 383 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:07.085352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171fab7-992d-4b5e-a753-8dfc89b6497c admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:07.087835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d171fab7-992d-4b5e-a753-8dfc89b6497c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:07.107322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1aa2949a-c125-4836-bafd-4633f0eb215e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b278c41191541e7bf182903fb9d2c14'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:07.136457 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 933/1858] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:06 2022] DELETE /identity/v3/users/9b278c41191541e7bf182903fb9d2c14 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:07.138006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2949a-c125-4836-bafd-4633f0eb215e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:07.139606 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1aa2949a-c125-4836-bafd-4633f0eb215e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:07.155522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:07.195471 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:07.207309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171fab7-992d-4b5e-a753-8dfc89b6497c admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:07.272505 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2949a-c125-4836-bafd-4633f0eb215e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:07.277696 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa8941-c0e5-4e9f-8188-1cef87da172d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:07.278330 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa8941-c0e5-4e9f-8188-1cef87da172d admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:07.278604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa8941-c0e5-4e9f-8188-1cef87da172d admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:07.278900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa8941-c0e5-4e9f-8188-1cef87da172d admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:07.279699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa8941-c0e5-4e9f-8188-1cef87da172d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:07.282081 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa8941-c0e5-4e9f-8188-1cef87da172d admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:07.315182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 934/1859] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:07 2022] POST /identity/v3/users => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:07.317342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083e6e3-45c8-4a07-9690-bbad098a19a4 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:07.317824 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083e6e3-45c8-4a07-9690-bbad098a19a4 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:07.318498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083e6e3-45c8-4a07-9690-bbad098a19a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:07.344903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e083e6e3-45c8-4a07-9690-bbad098a19a4 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:07.346195 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e083e6e3-45c8-4a07-9690-bbad098a19a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:07.347778 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:07.348817 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:07.349159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:07.349716 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c admin admin] PATH_INFO: `/v3/projects/6b193466c8384de5a9c7152d5c584ec4` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:07.353193 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b193466c8384de5a9c7152d5c584ec4)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:07.360474 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 935/1860] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:07.361811 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083e6e3-45c8-4a07-9690-bbad098a19a4 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:07.373295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083e6e3-45c8-4a07-9690-bbad098a19a4 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:07.376430 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:07.379534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc32de2-a39e-4048-9ec4-4114a9be3de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:07.380886 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc32de2-a39e-4048-9ec4-4114a9be3de4 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:07.381445 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc32de2-a39e-4048-9ec4-4114a9be3de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:07.381644 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc32de2-a39e-4048-9ec4-4114a9be3de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:07.382445 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc32de2-a39e-4048-9ec4-4114a9be3de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:07.384570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc32de2-a39e-4048-9ec4-4114a9be3de4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:07.413713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 936/1861] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:07.414818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc32de2-a39e-4048-9ec4-4114a9be3de4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:07.420885 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4cc32de2-a39e-4048-9ec4-4114a9be3de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:07.456136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b193466c8384de5a9c7152d5c584ec4', 'request_id': 'req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:07.464842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b193466c8384de5a9c7152d5c584ec4', 'request_id': 'req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:07.477379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b193466c8384de5a9c7152d5c584ec4', 'request_id': 'req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:07.499516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc32de2-a39e-4048-9ec4-4114a9be3de4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:07.501201 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c admin admin] The token cache is being invalidate because project 6b193466c8384de5a9c7152d5c584ec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:07.501201 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:07.502247 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:07.506020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 926/1862] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:07 2022] DELETE /identity/v3/projects/6b193466c8384de5a9c7152d5c584ec4 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:07.507353 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:07.509516 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:07.570409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176d8a3-971e-4eaa-9caa-e115cea63773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:07.570941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176d8a3-971e-4eaa-9caa-e115cea63773 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:07.571217 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176d8a3-971e-4eaa-9caa-e115cea63773 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:07.571525 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176d8a3-971e-4eaa-9caa-e115cea63773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:07.573621 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9176d8a3-971e-4eaa-9caa-e115cea63773 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:07.616111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648dbd9-90a3-42b3-8344-7757ddb3dd5c admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:07.630767 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44955bdb-d507-470b-a63e-28c44d0a323f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:07.631985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44955bdb-d507-470b-a63e-28c44d0a323f admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:07.632516 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44955bdb-d507-470b-a63e-28c44d0a323f admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:07.633191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44955bdb-d507-470b-a63e-28c44d0a323f admin admin] PATH_INFO: `/v3/projects/7b47b9b2c4914c0aac94509750990b16/users/91778d6bee3c402fb7a144045ebc64a5/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:07.653095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44955bdb-d507-470b-a63e-28c44d0a323f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b47b9b2c4914c0aac94509750990b16, user_id=91778d6bee3c402fb7a144045ebc64a5, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:07.655665 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:07.655980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44955bdb-d507-470b-a63e-28c44d0a323f admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:07.674200 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 927/1863] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:07 2022] PUT /identity/v3/projects/7b47b9b2c4914c0aac94509750990b16/users/91778d6bee3c402fb7a144045ebc64a5/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:07.675600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996f8f5-0c73-43b2-9d3e-de46137d0d31 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:07.675950 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996f8f5-0c73-43b2-9d3e-de46137d0d31 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:07.676311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996f8f5-0c73-43b2-9d3e-de46137d0d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:07.683184 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176d8a3-971e-4eaa-9caa-e115cea63773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:07.684739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176d8a3-971e-4eaa-9caa-e115cea63773 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:07.700365 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 937/1864] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:07.702640 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176d8a3-971e-4eaa-9caa-e115cea63773 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:07.704635 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-f996f8f5-0c73-43b2-9d3e-de46137d0d31 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:07.707183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f996f8f5-0c73-43b2-9d3e-de46137d0d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:07.715237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176d8a3-971e-4eaa-9caa-e115cea63773 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:07.740265 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:07.817900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ef673-3051-4296-b952-f1f7f2cfc550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:07.818733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ef673-3051-4296-b952-f1f7f2cfc550 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:07.819010 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ef673-3051-4296-b952-f1f7f2cfc550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:07.819281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ef673-3051-4296-b952-f1f7f2cfc550 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:07.820230 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ef673-3051-4296-b952-f1f7f2cfc550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:07.823481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ef673-3051-4296-b952-f1f7f2cfc550 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:07.848066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 938/1865] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:07.850106 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ef673-3051-4296-b952-f1f7f2cfc550 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:07.865058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 928/1866] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:07.868944 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ef673-3051-4296-b952-f1f7f2cfc550 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:07.873792 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d23e4d-67ba-458f-bfc7-00ff7b396183 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:07.874033 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d23e4d-67ba-458f-bfc7-00ff7b396183 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:07.874324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d23e4d-67ba-458f-bfc7-00ff7b396183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:07.875335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5540d6d-5f7a-4cbd-806b-3e81e4d71bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:07.876365 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5540d6d-5f7a-4cbd-806b-3e81e4d71bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:07.876689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5540d6d-5f7a-4cbd-806b-3e81e4d71bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:07.877034 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5540d6d-5f7a-4cbd-806b-3e81e4d71bd1 admin admin] PATH_INFO: `/v3/projects/7b47b9b2c4914c0aac94509750990b16/users/91778d6bee3c402fb7a144045ebc64a5/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:07.879292 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5540d6d-5f7a-4cbd-806b-3e81e4d71bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b47b9b2c4914c0aac94509750990b16, user_id=91778d6bee3c402fb7a144045ebc64a5, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:07.882257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5540d6d-5f7a-4cbd-806b-3e81e4d71bd1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:07.896287 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 939/1867] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:07 2022] PUT /identity/v3/projects/7b47b9b2c4914c0aac94509750990b16/users/91778d6bee3c402fb7a144045ebc64a5/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:07.908334 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923ae04-693c-404a-b3af-fc84c2a1d11e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:07.908643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923ae04-693c-404a-b3af-fc84c2a1d11e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:07.908878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923ae04-693c-404a-b3af-fc84c2a1d11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:07.927092 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-05d23e4d-67ba-458f-bfc7-00ff7b396183 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:07.930615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-05d23e4d-67ba-458f-bfc7-00ff7b396183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:07.949764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f923ae04-693c-404a-b3af-fc84c2a1d11e None None] MFA Rules not processed for user `91778d6bee3c402fb7a144045ebc64a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:07.951227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f923ae04-693c-404a-b3af-fc84c2a1d11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:08.034356 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 929/1868] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:08.039136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d23e4d-67ba-458f-bfc7-00ff7b396183 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:08.054401 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d23e4d-67ba-458f-bfc7-00ff7b396183 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:08.062064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716af42e-adb0-4e8f-b3ca-4391ceea2537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:08.062986 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716af42e-adb0-4e8f-b3ca-4391ceea2537 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:08.063580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716af42e-adb0-4e8f-b3ca-4391ceea2537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:08.065346 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716af42e-adb0-4e8f-b3ca-4391ceea2537 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:08.066346 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716af42e-adb0-4e8f-b3ca-4391ceea2537 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:45:08.067651 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716af42e-adb0-4e8f-b3ca-4391ceea2537 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:08.070202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716af42e-adb0-4e8f-b3ca-4391ceea2537 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:08.087286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 930/1869] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:45:08.092082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716af42e-adb0-4e8f-b3ca-4391ceea2537 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:08.105493 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 940/1870] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:08.110060 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716af42e-adb0-4e8f-b3ca-4391ceea2537 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:08.114564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923ae04-693c-404a-b3af-fc84c2a1d11e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:08.115447 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93348f29-1a5d-415a-b882-f554c00c05db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:08.116470 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93348f29-1a5d-415a-b882-f554c00c05db admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:08.116742 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93348f29-1a5d-415a-b882-f554c00c05db admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:08.117016 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93348f29-1a5d-415a-b882-f554c00c05db admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:08.117751 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93348f29-1a5d-415a-b882-f554c00c05db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:08.120089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93348f29-1a5d-415a-b882-f554c00c05db admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:08.120794 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f923ae04-693c-404a-b3af-fc84c2a1d11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:08.163581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 931/1871] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:08 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:08.168378 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93348f29-1a5d-415a-b882-f554c00c05db admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:08.294042 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93348f29-1a5d-415a-b882-f554c00c05db admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:08.309304 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7f0c4-9316-4512-b6b5-fea7ee5e33ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:08.310729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7f0c4-9316-4512-b6b5-fea7ee5e33ac admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:08.311278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7f0c4-9316-4512-b6b5-fea7ee5e33ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:08.312201 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7f0c4-9316-4512-b6b5-fea7ee5e33ac admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:08.313350 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7f0c4-9316-4512-b6b5-fea7ee5e33ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:08.314740 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923ae04-693c-404a-b3af-fc84c2a1d11e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:08.328810 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:08.329463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7f0c4-9316-4512-b6b5-fea7ee5e33ac admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:08.381205 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 932/1872] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:08 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:08.381924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7f0c4-9316-4512-b6b5-fea7ee5e33ac admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:08.385082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-34c7f0c4-9316-4512-b6b5-fea7ee5e33ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:08.407169 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7f0c4-9316-4512-b6b5-fea7ee5e33ac admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:08.427024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:08.428125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:08.428492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:08.429056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5 admin admin] PATH_INFO: `/v3/projects/6476c04596df43f4ba7c0af42ff340f2` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:08.430451 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6476c04596df43f4ba7c0af42ff340f2)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:08.433129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:08.447299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:08.528471 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00437387-f4c9-4140-b078-2867107d8189 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:08.529271 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00437387-f4c9-4140-b078-2867107d8189 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:08.529875 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00437387-f4c9-4140-b078-2867107d8189 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:08.529875 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00437387-f4c9-4140-b078-2867107d8189 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:08.531302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6476c04596df43f4ba7c0af42ff340f2', 'request_id': 'req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:08.538202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6476c04596df43f4ba7c0af42ff340f2', 'request_id': 'req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:08.547866 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6476c04596df43f4ba7c0af42ff340f2', 'request_id': 'req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:08.582322 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5 admin admin] The token cache is being invalidate because project 6476c04596df43f4ba7c0af42ff340f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:08.582322 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:08.582322 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:08.585558 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 933/1873] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:08 2022] DELETE /identity/v3/projects/6476c04596df43f4ba7c0af42ff340f2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:08.586724 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:08.588437 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:08.638686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00437387-f4c9-4140-b078-2867107d8189 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:08.639533 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00437387-f4c9-4140-b078-2867107d8189 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:08.663117 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 941/1874] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:45:08.664873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00437387-f4c9-4140-b078-2867107d8189 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:08.688050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00437387-f4c9-4140-b078-2867107d8189 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:08.709762 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c66de0-db59-4cf7-82fc-c5e8024f7fa5 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:08.717485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f47d7d-b2ac-43fc-a188-347e8b88804d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:08.718074 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f47d7d-b2ac-43fc-a188-347e8b88804d admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:08.718224 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f47d7d-b2ac-43fc-a188-347e8b88804d admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:08.718481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f47d7d-b2ac-43fc-a188-347e8b88804d admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:08.719119 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f47d7d-b2ac-43fc-a188-347e8b88804d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:08.721171 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f47d7d-b2ac-43fc-a188-347e8b88804d admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:08.737163 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 934/1875] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:08.738551 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f47d7d-b2ac-43fc-a188-347e8b88804d admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:08.750591 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f47d7d-b2ac-43fc-a188-347e8b88804d admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:08.756549 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774b861-4ecb-47eb-8093-a8b5984ead4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:08.758002 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774b861-4ecb-47eb-8093-a8b5984ead4d admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:08.758002 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774b861-4ecb-47eb-8093-a8b5984ead4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:08.758002 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774b861-4ecb-47eb-8093-a8b5984ead4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:08.758594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774b861-4ecb-47eb-8093-a8b5984ead4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:08.760052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774b861-4ecb-47eb-8093-a8b5984ead4d admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:08.773637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 935/1876] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:08.778164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774b861-4ecb-47eb-8093-a8b5984ead4d admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:08.789160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774b861-4ecb-47eb-8093-a8b5984ead4d admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:08.799167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56c948-8bc0-4cd4-abad-8c8560cb6be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:08.800894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56c948-8bc0-4cd4-abad-8c8560cb6be9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:08.801903 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56c948-8bc0-4cd4-abad-8c8560cb6be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:08.802543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56c948-8bc0-4cd4-abad-8c8560cb6be9 admin admin] PATH_INFO: `/v3/projects/a98c4c86a6c54cacb4b22668ce01ca93/users/81ba430b51dc4f379d957ae0ff0d117d/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:08.854257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56c948-8bc0-4cd4-abad-8c8560cb6be9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a98c4c86a6c54cacb4b22668ce01ca93, user_id=81ba430b51dc4f379d957ae0ff0d117d, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:08.857364 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56c948-8bc0-4cd4-abad-8c8560cb6be9 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:08.885028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ef73b-200e-4e54-8d53-7a96db7f128c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:08.886071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ef73b-200e-4e54-8d53-7a96db7f128c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:08.886530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ef73b-200e-4e54-8d53-7a96db7f128c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:08.887023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ef73b-200e-4e54-8d53-7a96db7f128c admin admin] PATH_INFO: `/v3/users/62c7d1cd4b7e4677a261f41ae315f399` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:08.889228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ef73b-200e-4e54-8d53-7a96db7f128c admin admin] RBAC: Authorizing `identity:delete_user(user_id=62c7d1cd4b7e4677a261f41ae315f399)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:08.892663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ef73b-200e-4e54-8d53-7a96db7f128c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:08.893520 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 936/1877] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:08 2022] PUT /identity/v3/projects/a98c4c86a6c54cacb4b22668ce01ca93/users/81ba430b51dc4f379d957ae0ff0d117d/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:08.903204 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56c948-8bc0-4cd4-abad-8c8560cb6be9 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:08.907231 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7d56c948-8bc0-4cd4-abad-8c8560cb6be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:08.981291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-eb3ef73b-200e-4e54-8d53-7a96db7f128c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62c7d1cd4b7e4677a261f41ae315f399'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:08.992964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-eb3ef73b-200e-4e54-8d53-7a96db7f128c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62c7d1cd4b7e4677a261f41ae315f399'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:08.995695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:08.996700 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:09.003625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56c948-8bc0-4cd4-abad-8c8560cb6be9 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:09.006842 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-eb3ef73b-200e-4e54-8d53-7a96db7f128c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62c7d1cd4b7e4677a261f41ae315f399'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:09.008137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-eb3ef73b-200e-4e54-8d53-7a96db7f128c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62c7d1cd4b7e4677a261f41ae315f399'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:09.012851 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d386f-1b42-4d30-b908-39b49deb6458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:09.013848 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d386f-1b42-4d30-b908-39b49deb6458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:09.014238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d386f-1b42-4d30-b908-39b49deb6458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:09.014684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d386f-1b42-4d30-b908-39b49deb6458 admin admin] PATH_INFO: `/v3/users/c7250085a1f940ab87a26c5ced51ca33` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:09.017704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d386f-1b42-4d30-b908-39b49deb6458 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7250085a1f940ab87a26c5ced51ca33)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:09.020424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d386f-1b42-4d30-b908-39b49deb6458 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:09.031307 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-eb3ef73b-200e-4e54-8d53-7a96db7f128c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62c7d1cd4b7e4677a261f41ae315f399'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:09.040858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 942/1878] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:08 2022] DELETE /identity/v3/users/62c7d1cd4b7e4677a261f41ae315f399 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:09.043211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571291a-abfb-4afd-9ed6-179dbae1230a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:09.043799 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571291a-abfb-4afd-9ed6-179dbae1230a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:09.044394 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571291a-abfb-4afd-9ed6-179dbae1230a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:09.089815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a571291a-abfb-4afd-9ed6-179dbae1230a None None] MFA Rules not processed for user `81ba430b51dc4f379d957ae0ff0d117d`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:09.091168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a571291a-abfb-4afd-9ed6-179dbae1230a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:09.096944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-7e6d386f-1b42-4d30-b908-39b49deb6458 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7250085a1f940ab87a26c5ced51ca33'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:09.097801 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-7e6d386f-1b42-4d30-b908-39b49deb6458 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7250085a1f940ab87a26c5ced51ca33'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:09.098483 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-7e6d386f-1b42-4d30-b908-39b49deb6458 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7250085a1f940ab87a26c5ced51ca33'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:09.109866 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-7e6d386f-1b42-4d30-b908-39b49deb6458 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7250085a1f940ab87a26c5ced51ca33'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:09.128205 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-7e6d386f-1b42-4d30-b908-39b49deb6458 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7250085a1f940ab87a26c5ced51ca33'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:09.146533 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 937/1879] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:08 2022] DELETE /identity/v3/users/c7250085a1f940ab87a26c5ced51ca33 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:09.151027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5c098-4173-4c94-9ac6-d89e239e7aed None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:09.151329 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5c098-4173-4c94-9ac6-d89e239e7aed None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:09.151978 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5c098-4173-4c94-9ac6-d89e239e7aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:09.177292 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-dcc5c098-4173-4c94-9ac6-d89e239e7aed None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:09.178755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-dcc5c098-4173-4c94-9ac6-d89e239e7aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:09.221312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 943/1880] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:09.235723 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571291a-abfb-4afd-9ed6-179dbae1230a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:09.238256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a571291a-abfb-4afd-9ed6-179dbae1230a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:09.337811 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 938/1881] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:09.370612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571291a-abfb-4afd-9ed6-179dbae1230a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:09.387141 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5c098-4173-4c94-9ac6-d89e239e7aed None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:09.389615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-dcc5c098-4173-4c94-9ac6-d89e239e7aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:09.520039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae625b-413d-4c36-999b-5492c77d9f03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:09.521095 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae625b-413d-4c36-999b-5492c77d9f03 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:09.526976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5c098-4173-4c94-9ac6-d89e239e7aed None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:09.531353 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f527410-924c-4342-82b2-35e6235018cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:09.532035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f527410-924c-4342-82b2-35e6235018cc service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:09.532301 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f527410-924c-4342-82b2-35e6235018cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:09.532574 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f527410-924c-4342-82b2-35e6235018cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:09.535068 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae625b-413d-4c36-999b-5492c77d9f03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:09.535961 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae625b-413d-4c36-999b-5492c77d9f03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:09.544739 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f527410-924c-4342-82b2-35e6235018cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:09.546430 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f527410-924c-4342-82b2-35e6235018cc service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:09.558108 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 939/1882] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:09.559120 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f527410-924c-4342-82b2-35e6235018cc service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:09.561228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7f527410-924c-4342-82b2-35e6235018cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:09.575794 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f527410-924c-4342-82b2-35e6235018cc service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:09.587554 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc393457-c6a7-42de-bd15-49ec915de64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:09.588257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc393457-c6a7-42de-bd15-49ec915de64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:09.588537 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc393457-c6a7-42de-bd15-49ec915de64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:09.588813 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc393457-c6a7-42de-bd15-49ec915de64e admin admin] PATH_INFO: `/v3/projects/8428008d569e4a0da2201825fdb9b8d4` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:09.590066 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc393457-c6a7-42de-bd15-49ec915de64e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8428008d569e4a0da2201825fdb9b8d4)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:09.592800 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc393457-c6a7-42de-bd15-49ec915de64e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:09.614915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:09.631481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:09.696377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae625b-413d-4c36-999b-5492c77d9f03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:09.698918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae625b-413d-4c36-999b-5492c77d9f03 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:09.699613 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-fc393457-c6a7-42de-bd15-49ec915de64e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8428008d569e4a0da2201825fdb9b8d4', 'request_id': 'req-fc393457-c6a7-42de-bd15-49ec915de64e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:09.715182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 944/1883] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:45:09.739699 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-fc393457-c6a7-42de-bd15-49ec915de64e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8428008d569e4a0da2201825fdb9b8d4', 'request_id': 'req-fc393457-c6a7-42de-bd15-49ec915de64e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:09.813308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-fc393457-c6a7-42de-bd15-49ec915de64e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8428008d569e4a0da2201825fdb9b8d4', 'request_id': 'req-fc393457-c6a7-42de-bd15-49ec915de64e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:09.869708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-fc393457-c6a7-42de-bd15-49ec915de64e admin admin] The token cache is being invalidate because project 8428008d569e4a0da2201825fdb9b8d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:09.871064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-fc393457-c6a7-42de-bd15-49ec915de64e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:09.874657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-fc393457-c6a7-42de-bd15-49ec915de64e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:09.881037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 940/1884] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:09 2022] DELETE /identity/v3/projects/8428008d569e4a0da2201825fdb9b8d4 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:09.886068 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4e431-c527-45b2-8ff0-bc6924ef7608 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:09.886418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4e431-c527-45b2-8ff0-bc6924ef7608 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:09.886721 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4e431-c527-45b2-8ff0-bc6924ef7608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:09.950361 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d2b4e431-c527-45b2-8ff0-bc6924ef7608 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:09.952382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d2b4e431-c527-45b2-8ff0-bc6924ef7608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:10.114269 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 945/1885] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:10.132365 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56d535-d319-45df-a795-2efeb07960e4 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:10.133270 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56d535-d319-45df-a795-2efeb07960e4 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:10.133786 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56d535-d319-45df-a795-2efeb07960e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:10.168123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-dd56d535-d319-45df-a795-2efeb07960e4 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:10.169536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-dd56d535-d319-45df-a795-2efeb07960e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:10.180053 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 941/1886] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:10.184584 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4e431-c527-45b2-8ff0-bc6924ef7608 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:10.196836 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4e431-c527-45b2-8ff0-bc6924ef7608 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:10.203826 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554be90-b72e-448e-a76c-d0c18c54d405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:10.204635 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554be90-b72e-448e-a76c-d0c18c54d405 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:10.204982 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554be90-b72e-448e-a76c-d0c18c54d405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:10.205173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554be90-b72e-448e-a76c-d0c18c54d405 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:10.205906 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554be90-b72e-448e-a76c-d0c18c54d405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:45:10.206258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554be90-b72e-448e-a76c-d0c18c54d405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:10.207929 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554be90-b72e-448e-a76c-d0c18c54d405 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:10.224788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 946/1887] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:45:10.229306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56d535-d319-45df-a795-2efeb07960e4 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:10.240980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56d535-d319-45df-a795-2efeb07960e4 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:10.249309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600b5f3-23f3-4c3c-a0f5-3e776e9c9f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:10.250059 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600b5f3-23f3-4c3c-a0f5-3e776e9c9f5a admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:10.250465 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600b5f3-23f3-4c3c-a0f5-3e776e9c9f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:10.250937 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600b5f3-23f3-4c3c-a0f5-3e776e9c9f5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:10.252056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600b5f3-23f3-4c3c-a0f5-3e776e9c9f5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:10.254293 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600b5f3-23f3-4c3c-a0f5-3e776e9c9f5a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:10.298157 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 942/1888] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:10 2022] POST /identity/v3/projects => generated 387 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:45:10.301268 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554be90-b72e-448e-a76c-d0c18c54d405 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:10.314305 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554be90-b72e-448e-a76c-d0c18c54d405 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:10.453199 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ed465-64ab-4a24-a57f-fd4fdf5676da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:10.459352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ed465-64ab-4a24-a57f-fd4fdf5676da admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:10.459754 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ed465-64ab-4a24-a57f-fd4fdf5676da admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:10.460218 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ed465-64ab-4a24-a57f-fd4fdf5676da admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:10.461285 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ed465-64ab-4a24-a57f-fd4fdf5676da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:10.468192 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ed465-64ab-4a24-a57f-fd4fdf5676da admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:10.522192 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 947/1889] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:10 2022] POST /identity/v3/users => generated 294 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:10.526778 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600b5f3-23f3-4c3c-a0f5-3e776e9c9f5a admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:10.542643 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600b5f3-23f3-4c3c-a0f5-3e776e9c9f5a admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:10.553252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19780ac-3b5f-4ff1-927c-597ed80a7f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:10.554250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19780ac-3b5f-4ff1-927c-597ed80a7f03 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:10.554738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19780ac-3b5f-4ff1-927c-597ed80a7f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:10.555696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19780ac-3b5f-4ff1-927c-597ed80a7f03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:10.556646 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19780ac-3b5f-4ff1-927c-597ed80a7f03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:10.561044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19780ac-3b5f-4ff1-927c-597ed80a7f03 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:10.579659 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 943/1890] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:10.582037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ed465-64ab-4a24-a57f-fd4fdf5676da admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:10.598862 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ed465-64ab-4a24-a57f-fd4fdf5676da admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:10.614262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b49c6-5a5a-4d54-8c29-5ec6c65ab418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:10.615279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b49c6-5a5a-4d54-8c29-5ec6c65ab418 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:10.615775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b49c6-5a5a-4d54-8c29-5ec6c65ab418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:10.616245 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b49c6-5a5a-4d54-8c29-5ec6c65ab418 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:10.617439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b49c6-5a5a-4d54-8c29-5ec6c65ab418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:10.621186 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b49c6-5a5a-4d54-8c29-5ec6c65ab418 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:10.632371 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19780ac-3b5f-4ff1-927c-597ed80a7f03 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:10.634927 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f19780ac-3b5f-4ff1-927c-597ed80a7f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:10.642391 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 948/1891] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:10.647109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b49c6-5a5a-4d54-8c29-5ec6c65ab418 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:10.666464 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b49c6-5a5a-4d54-8c29-5ec6c65ab418 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:10.670978 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19780ac-3b5f-4ff1-927c-597ed80a7f03 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:10.673055 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da26a2-6938-40e4-b1f9-f98be46dc544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:10.674294 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da26a2-6938-40e4-b1f9-f98be46dc544 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:10.675048 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da26a2-6938-40e4-b1f9-f98be46dc544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:10.675531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da26a2-6938-40e4-b1f9-f98be46dc544 admin admin] PATH_INFO: `/v3/projects/bc5ac77baea4479facf1af48f5b7dbf1/users/a5fa1023a13f4c0aad4a83d0a53a25d9/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:10.681473 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cdfab-e9c9-45c8-93c2-3df99fd888b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:10.682510 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cdfab-e9c9-45c8-93c2-3df99fd888b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:10.683236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cdfab-e9c9-45c8-93c2-3df99fd888b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:10.683876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cdfab-e9c9-45c8-93c2-3df99fd888b8 admin admin] PATH_INFO: `/v3/projects/747547b986f1448ca0e9e43a76906a0f` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:10.685666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cdfab-e9c9-45c8-93c2-3df99fd888b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=747547b986f1448ca0e9e43a76906a0f)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:10.688899 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cdfab-e9c9-45c8-93c2-3df99fd888b8 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:10.701894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da26a2-6938-40e4-b1f9-f98be46dc544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc5ac77baea4479facf1af48f5b7dbf1, user_id=a5fa1023a13f4c0aad4a83d0a53a25d9, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:10.705287 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da26a2-6938-40e4-b1f9-f98be46dc544 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:10.736123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 949/1892] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:10 2022] PUT /identity/v3/projects/bc5ac77baea4479facf1af48f5b7dbf1/users/a5fa1023a13f4c0aad4a83d0a53a25d9/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:10.741378 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225f8ad-4ffd-4ef6-8c0e-a394b9564795 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:10.741704 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225f8ad-4ffd-4ef6-8c0e-a394b9564795 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:10.741933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225f8ad-4ffd-4ef6-8c0e-a394b9564795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:10.802194 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-103cdfab-e9c9-45c8-93c2-3df99fd888b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '747547b986f1448ca0e9e43a76906a0f', 'request_id': 'req-103cdfab-e9c9-45c8-93c2-3df99fd888b8'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:10.806191 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0225f8ad-4ffd-4ef6-8c0e-a394b9564795 None None] MFA Rules not processed for user `a5fa1023a13f4c0aad4a83d0a53a25d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:10.807675 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0225f8ad-4ffd-4ef6-8c0e-a394b9564795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:10.824680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-103cdfab-e9c9-45c8-93c2-3df99fd888b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '747547b986f1448ca0e9e43a76906a0f', 'request_id': 'req-103cdfab-e9c9-45c8-93c2-3df99fd888b8'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:10.825486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:10.844300 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-103cdfab-e9c9-45c8-93c2-3df99fd888b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '747547b986f1448ca0e9e43a76906a0f', 'request_id': 'req-103cdfab-e9c9-45c8-93c2-3df99fd888b8'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:10.873404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-103cdfab-e9c9-45c8-93c2-3df99fd888b8 admin admin] The token cache is being invalidate because project 747547b986f1448ca0e9e43a76906a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:10.876748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-103cdfab-e9c9-45c8-93c2-3df99fd888b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:10.878445 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-103cdfab-e9c9-45c8-93c2-3df99fd888b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:10.884548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 944/1893] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:10 2022] DELETE /identity/v3/projects/747547b986f1448ca0e9e43a76906a0f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:10.884548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cdfab-e9c9-45c8-93c2-3df99fd888b8 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:10.895398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-103cdfab-e9c9-45c8-93c2-3df99fd888b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:10.981710 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 950/1894] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:10.983820 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225f8ad-4ffd-4ef6-8c0e-a394b9564795 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:10.987676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0225f8ad-4ffd-4ef6-8c0e-a394b9564795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:10.990687 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:11.038011 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cdfab-e9c9-45c8-93c2-3df99fd888b8 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:11.049724 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfbc86-12fa-4c3b-884e-81efd2527d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:11.051278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfbc86-12fa-4c3b-884e-81efd2527d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.051690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfbc86-12fa-4c3b-884e-81efd2527d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.052230 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfbc86-12fa-4c3b-884e-81efd2527d9a admin admin] PATH_INFO: `/v3/users/72ec3418de51407fa670e959e288cc64` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.057373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfbc86-12fa-4c3b-884e-81efd2527d9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=72ec3418de51407fa670e959e288cc64)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:11.065326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfbc86-12fa-4c3b-884e-81efd2527d9a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:11.147540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225f8ad-4ffd-4ef6-8c0e-a394b9564795 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:11.154314 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e743633-d9e0-46e4-8828-c1a7e51cf7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:11.155135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e743633-d9e0-46e4-8828-c1a7e51cf7e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.155419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e743633-d9e0-46e4-8828-c1a7e51cf7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.155689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e743633-d9e0-46e4-8828-c1a7e51cf7e8 admin admin] PATH_INFO: `/v3/users/91778d6bee3c402fb7a144045ebc64a5` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.157052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e743633-d9e0-46e4-8828-c1a7e51cf7e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91778d6bee3c402fb7a144045ebc64a5)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:11.159176 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e743633-d9e0-46e4-8828-c1a7e51cf7e8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:11.170634 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f6dfbc86-12fa-4c3b-884e-81efd2527d9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72ec3418de51407fa670e959e288cc64'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:11.171569 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f6dfbc86-12fa-4c3b-884e-81efd2527d9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72ec3418de51407fa670e959e288cc64'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:11.172335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f6dfbc86-12fa-4c3b-884e-81efd2527d9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72ec3418de51407fa670e959e288cc64'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:11.183612 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f6dfbc86-12fa-4c3b-884e-81efd2527d9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72ec3418de51407fa670e959e288cc64'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:11.208939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f6dfbc86-12fa-4c3b-884e-81efd2527d9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72ec3418de51407fa670e959e288cc64'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:11.230952 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4e743633-d9e0-46e4-8828-c1a7e51cf7e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91778d6bee3c402fb7a144045ebc64a5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:11.236128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 945/1895] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:10 2022] DELETE /identity/v3/users/72ec3418de51407fa670e959e288cc64 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:11.236128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfbc86-12fa-4c3b-884e-81efd2527d9a admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:11.237191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f6dfbc86-12fa-4c3b-884e-81efd2527d9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:11.243650 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4e743633-d9e0-46e4-8828-c1a7e51cf7e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91778d6bee3c402fb7a144045ebc64a5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:11.274415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4e743633-d9e0-46e4-8828-c1a7e51cf7e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91778d6bee3c402fb7a144045ebc64a5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:11.275485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4e743633-d9e0-46e4-8828-c1a7e51cf7e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91778d6bee3c402fb7a144045ebc64a5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:11.294671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4e743633-d9e0-46e4-8828-c1a7e51cf7e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91778d6bee3c402fb7a144045ebc64a5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:11.306469 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 951/1896] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:10 2022] DELETE /identity/v3/users/91778d6bee3c402fb7a144045ebc64a5 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:11.306469 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaddbc-dfc2-4a20-b918-c5e284dd974b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.307118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaddbc-dfc2-4a20-b918-c5e284dd974b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.307511 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaddbc-dfc2-4a20-b918-c5e284dd974b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.345690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c1eaddbc-dfc2-4a20-b918-c5e284dd974b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:11.347022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c1eaddbc-dfc2-4a20-b918-c5e284dd974b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:11.392964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:11.404504 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfbc86-12fa-4c3b-884e-81efd2527d9a admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:11.466193 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 952/1897] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:11.478652 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eaddbc-dfc2-4a20-b918-c5e284dd974b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:11.480772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c1eaddbc-dfc2-4a20-b918-c5e284dd974b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:11.524595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a5f95-f7d9-482d-929d-6cfc34fd1e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:11.525278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a5f95-f7d9-482d-929d-6cfc34fd1e07 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.525591 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a5f95-f7d9-482d-929d-6cfc34fd1e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.525858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a5f95-f7d9-482d-929d-6cfc34fd1e07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.526631 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a5f95-f7d9-482d-929d-6cfc34fd1e07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:11.529299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a5f95-f7d9-482d-929d-6cfc34fd1e07 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:11.557375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:11.570523 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 946/1898] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:11 2022] POST /identity/v3/projects => generated 387 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:11.575282 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a5f95-f7d9-482d-929d-6cfc34fd1e07 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:11.588052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b78a5f95-f7d9-482d-929d-6cfc34fd1e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:11.588528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eaddbc-dfc2-4a20-b918-c5e284dd974b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:11.672152 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ea28e-8417-4f82-bd94-2d289e5c221b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:11.673047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ea28e-8417-4f82-bd94-2d289e5c221b service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.673371 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ea28e-8417-4f82-bd94-2d289e5c221b service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.673587 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ea28e-8417-4f82-bd94-2d289e5c221b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.715795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a5f95-f7d9-482d-929d-6cfc34fd1e07 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:11.726115 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692e500-1828-4ba4-a0e5-2d5c845d4448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:11.727766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692e500-1828-4ba4-a0e5-2d5c845d4448 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.729010 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692e500-1828-4ba4-a0e5-2d5c845d4448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.729296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692e500-1828-4ba4-a0e5-2d5c845d4448 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.729976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692e500-1828-4ba4-a0e5-2d5c845d4448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:11.732065 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692e500-1828-4ba4-a0e5-2d5c845d4448 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:11.764070 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 947/1899] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:11 2022] POST /identity/v3/users => generated 300 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:11.768981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692e500-1828-4ba4-a0e5-2d5c845d4448 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:11.789174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692e500-1828-4ba4-a0e5-2d5c845d4448 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:11.798400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57885344-c423-4d3d-9635-e0b1135f3d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:11.799436 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57885344-c423-4d3d-9635-e0b1135f3d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.800294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57885344-c423-4d3d-9635-e0b1135f3d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.800537 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57885344-c423-4d3d-9635-e0b1135f3d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.801730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57885344-c423-4d3d-9635-e0b1135f3d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:11.806125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57885344-c423-4d3d-9635-e0b1135f3d63 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:11.815035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ea28e-8417-4f82-bd94-2d289e5c221b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:11.816552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ea28e-8417-4f82-bd94-2d289e5c221b service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:11.830244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 948/1901] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:45:11.832140 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 953/1901] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:45:11.835927 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57885344-c423-4d3d-9635-e0b1135f3d63 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:11.857367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57885344-c423-4d3d-9635-e0b1135f3d63 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:11.866089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f299af-d3fa-4ae6-b336-d21f1164ccab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:11.867124 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f299af-d3fa-4ae6-b336-d21f1164ccab admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.867976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f299af-d3fa-4ae6-b336-d21f1164ccab admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.868453 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f299af-d3fa-4ae6-b336-d21f1164ccab admin admin] PATH_INFO: `/v3/projects/d8f40e0afaa24f40bb7c4112864a7bf3/users/0321d1017e1d4672851f0693491798a9/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.889017 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f299af-d3fa-4ae6-b336-d21f1164ccab admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f40e0afaa24f40bb7c4112864a7bf3, user_id=0321d1017e1d4672851f0693491798a9, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:11.892057 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f299af-d3fa-4ae6-b336-d21f1164ccab admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:11.909994 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 949/1902] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:11 2022] PUT /identity/v3/projects/d8f40e0afaa24f40bb7c4112864a7bf3/users/0321d1017e1d4672851f0693491798a9/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:11.916231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ea28e-8417-4f82-bd94-2d289e5c221b service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:11.931703 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ea28e-8417-4f82-bd94-2d289e5c221b service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:11.973495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:12.019248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0430d3a-c129-4e96-a96a-eb93228cf55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:12.019792 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0430d3a-c129-4e96-a96a-eb93228cf55d admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:12.020113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0430d3a-c129-4e96-a96a-eb93228cf55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:12.020386 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0430d3a-c129-4e96-a96a-eb93228cf55d admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:12.021056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0430d3a-c129-4e96-a96a-eb93228cf55d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:12.022709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0430d3a-c129-4e96-a96a-eb93228cf55d admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:12.041567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 954/1903] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:12.046234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f299af-d3fa-4ae6-b336-d21f1164ccab admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:12.066681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f299af-d3fa-4ae6-b336-d21f1164ccab admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:12.073462 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30843749-33fa-443e-b89e-871e02509956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:12.074208 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30843749-33fa-443e-b89e-871e02509956 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:12.074451 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30843749-33fa-443e-b89e-871e02509956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:12.074723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30843749-33fa-443e-b89e-871e02509956 admin admin] PATH_INFO: `/v3/projects/d8f40e0afaa24f40bb7c4112864a7bf3/users/0321d1017e1d4672851f0693491798a9/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:12.077171 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30843749-33fa-443e-b89e-871e02509956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f40e0afaa24f40bb7c4112864a7bf3, user_id=0321d1017e1d4672851f0693491798a9, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:12.079952 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30843749-33fa-443e-b89e-871e02509956 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:12.095330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 950/1904] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:12 2022] PUT /identity/v3/projects/d8f40e0afaa24f40bb7c4112864a7bf3/users/0321d1017e1d4672851f0693491798a9/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:12.114774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c239e52-bc2e-48ba-a8b7-ab5d5c5cd4c8 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:12.116249 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c239e52-bc2e-48ba-a8b7-ab5d5c5cd4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:12.116578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c239e52-bc2e-48ba-a8b7-ab5d5c5cd4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:12.117780 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30843749-33fa-443e-b89e-871e02509956 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:12.120756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-30843749-33fa-443e-b89e-871e02509956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:12.159907 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2c239e52-bc2e-48ba-a8b7-ab5d5c5cd4c8 None None] MFA Rules not processed for user `0321d1017e1d4672851f0693491798a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:12.166575 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2c239e52-bc2e-48ba-a8b7-ab5d5c5cd4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:12.291860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30843749-33fa-443e-b89e-871e02509956 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:12.301555 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552a26a-8f18-4924-bf0f-0a4f07b2789c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:12.302812 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552a26a-8f18-4924-bf0f-0a4f07b2789c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:12.303300 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552a26a-8f18-4924-bf0f-0a4f07b2789c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:12.304131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552a26a-8f18-4924-bf0f-0a4f07b2789c admin admin] PATH_INFO: `/v3/projects/7b47b9b2c4914c0aac94509750990b16` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:12.305911 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552a26a-8f18-4924-bf0f-0a4f07b2789c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b47b9b2c4914c0aac94509750990b16)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:12.309708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552a26a-8f18-4924-bf0f-0a4f07b2789c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:12.361869 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 955/1905] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:12.374462 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c239e52-bc2e-48ba-a8b7-ab5d5c5cd4c8 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:12.377282 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2c239e52-bc2e-48ba-a8b7-ab5d5c5cd4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:12.407983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:12.426076 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8552a26a-8f18-4924-bf0f-0a4f07b2789c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b47b9b2c4914c0aac94509750990b16', 'request_id': 'req-8552a26a-8f18-4924-bf0f-0a4f07b2789c'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:12.445171 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8552a26a-8f18-4924-bf0f-0a4f07b2789c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b47b9b2c4914c0aac94509750990b16', 'request_id': 'req-8552a26a-8f18-4924-bf0f-0a4f07b2789c'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:12.454863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8552a26a-8f18-4924-bf0f-0a4f07b2789c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b47b9b2c4914c0aac94509750990b16', 'request_id': 'req-8552a26a-8f18-4924-bf0f-0a4f07b2789c'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:12.479318 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8552a26a-8f18-4924-bf0f-0a4f07b2789c admin admin] The token cache is being invalidate because project 7b47b9b2c4914c0aac94509750990b16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:12.479318 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8552a26a-8f18-4924-bf0f-0a4f07b2789c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:12.481351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8552a26a-8f18-4924-bf0f-0a4f07b2789c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:12.485601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 951/1906] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:12 2022] DELETE /identity/v3/projects/7b47b9b2c4914c0aac94509750990b16 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:12.491740 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92259f9a-be7f-4950-8055-80dc0a2ccbdf None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:12.492252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92259f9a-be7f-4950-8055-80dc0a2ccbdf None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:12.492747 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92259f9a-be7f-4950-8055-80dc0a2ccbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:12.540354 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-92259f9a-be7f-4950-8055-80dc0a2ccbdf None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:12.542308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-92259f9a-be7f-4950-8055-80dc0a2ccbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:12.542617 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c239e52-bc2e-48ba-a8b7-ab5d5c5cd4c8 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:12.661189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0aef41-0ab9-44f3-a719-72514061f2b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:12.661990 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0aef41-0ab9-44f3-a719-72514061f2b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:12.662522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0aef41-0ab9-44f3-a719-72514061f2b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:12.662522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0aef41-0ab9-44f3-a719-72514061f2b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:12.667558 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5a0aef41-0ab9-44f3-a719-72514061f2b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:12.688713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 952/1907] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:12.690283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92259f9a-be7f-4950-8055-80dc0a2ccbdf None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:12.696728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-92259f9a-be7f-4950-8055-80dc0a2ccbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:12.711800 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:12.719477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92259f9a-be7f-4950-8055-80dc0a2ccbdf None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:12.725385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ceead-d04f-4169-8733-f34e3af1f299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:12.726066 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ceead-d04f-4169-8733-f34e3af1f299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:12.726368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ceead-d04f-4169-8733-f34e3af1f299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:12.726742 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ceead-d04f-4169-8733-f34e3af1f299 admin admin] PATH_INFO: `/v3/projects/7ec055a45c5a4a0d9a763daceb8131e9` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:12.728122 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ceead-d04f-4169-8733-f34e3af1f299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec055a45c5a4a0d9a763daceb8131e9)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:12.730088 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ceead-d04f-4169-8733-f34e3af1f299 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:12.772690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0aef41-0ab9-44f3-a719-72514061f2b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:12.774173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0aef41-0ab9-44f3-a719-72514061f2b5 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:12.787545 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 956/1908] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:12.788842 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ef3a1-0c54-4905-adf0-d26424d78e74 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:12.789078 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ef3a1-0c54-4905-adf0-d26424d78e74 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:12.789361 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ef3a1-0c54-4905-adf0-d26424d78e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:12.805825 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-168ceead-d04f-4169-8733-f34e3af1f299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec055a45c5a4a0d9a763daceb8131e9', 'request_id': 'req-168ceead-d04f-4169-8733-f34e3af1f299'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:12.818033 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-168ceead-d04f-4169-8733-f34e3af1f299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec055a45c5a4a0d9a763daceb8131e9', 'request_id': 'req-168ceead-d04f-4169-8733-f34e3af1f299'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:12.820676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-b20ef3a1-0c54-4905-adf0-d26424d78e74 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:12.822018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b20ef3a1-0c54-4905-adf0-d26424d78e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:12.832292 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-168ceead-d04f-4169-8733-f34e3af1f299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec055a45c5a4a0d9a763daceb8131e9', 'request_id': 'req-168ceead-d04f-4169-8733-f34e3af1f299'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:12.852177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-168ceead-d04f-4169-8733-f34e3af1f299 admin admin] The token cache is being invalidate because project 7ec055a45c5a4a0d9a763daceb8131e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:12.852474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-168ceead-d04f-4169-8733-f34e3af1f299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:12.854401 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-168ceead-d04f-4169-8733-f34e3af1f299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:12.860502 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 953/1909] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:12 2022] DELETE /identity/v3/projects/7ec055a45c5a4a0d9a763daceb8131e9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:12.866260 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ceead-d04f-4169-8733-f34e3af1f299 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:12.868027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-168ceead-d04f-4169-8733-f34e3af1f299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:12.926867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 957/1910] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:12.931921 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ef3a1-0c54-4905-adf0-d26424d78e74 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:12.942958 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ef3a1-0c54-4905-adf0-d26424d78e74 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:12.945497 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:12.950237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fca53f-8405-4e4a-a6a0-fcc7195e477f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:12.951217 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fca53f-8405-4e4a-a6a0-fcc7195e477f admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:12.951604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fca53f-8405-4e4a-a6a0-fcc7195e477f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:12.952440 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fca53f-8405-4e4a-a6a0-fcc7195e477f admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:12.953176 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fca53f-8405-4e4a-a6a0-fcc7195e477f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:45:12.953478 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fca53f-8405-4e4a-a6a0-fcc7195e477f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:12.955119 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fca53f-8405-4e4a-a6a0-fcc7195e477f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:12.969807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 958/1911] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:45:12.974530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ceead-d04f-4169-8733-f34e3af1f299 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:12.976613 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fca53f-8405-4e4a-a6a0-fcc7195e477f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:12.980665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222024d-0b09-4e3a-805d-272be8818dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:12.981337 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222024d-0b09-4e3a-805d-272be8818dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:12.981613 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222024d-0b09-4e3a-805d-272be8818dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:12.981935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222024d-0b09-4e3a-805d-272be8818dbe admin admin] PATH_INFO: `/v3/users/c7d59b2f5c944b92b37360ecc88d84ca` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:12.983333 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222024d-0b09-4e3a-805d-272be8818dbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7d59b2f5c944b92b37360ecc88d84ca)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:12.985481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222024d-0b09-4e3a-805d-272be8818dbe admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:12.990925 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fca53f-8405-4e4a-a6a0-fcc7195e477f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:12.998830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32e8d1-8ab3-4e20-8ad7-c0acedbf80c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:12.999426 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32e8d1-8ab3-4e20-8ad7-c0acedbf80c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:12.999589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32e8d1-8ab3-4e20-8ad7-c0acedbf80c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:13.000692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32e8d1-8ab3-4e20-8ad7-c0acedbf80c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:13.001360 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e32e8d1-8ab3-4e20-8ad7-c0acedbf80c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:13.004149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e32e8d1-8ab3-4e20-8ad7-c0acedbf80c0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:13.040038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 959/1912] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:12 2022] POST /identity/v3/projects => generated 401 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:45:13.046283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-1222024d-0b09-4e3a-805d-272be8818dbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7d59b2f5c944b92b37360ecc88d84ca'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:13.046874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32e8d1-8ab3-4e20-8ad7-c0acedbf80c0 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:13.046985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-1222024d-0b09-4e3a-805d-272be8818dbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7d59b2f5c944b92b37360ecc88d84ca'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:13.047652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-1222024d-0b09-4e3a-805d-272be8818dbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7d59b2f5c944b92b37360ecc88d84ca'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:13.049700 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9e32e8d1-8ab3-4e20-8ad7-c0acedbf80c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:13.056894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-1222024d-0b09-4e3a-805d-272be8818dbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7d59b2f5c944b92b37360ecc88d84ca'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:13.073601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-1222024d-0b09-4e3a-805d-272be8818dbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7d59b2f5c944b92b37360ecc88d84ca'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:13.094967 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 954/1913] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:12 2022] DELETE /identity/v3/users/c7d59b2f5c944b92b37360ecc88d84ca => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:13.181786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32e8d1-8ab3-4e20-8ad7-c0acedbf80c0 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:13.228782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:13.276692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c44a2-9a80-4a3a-adf7-0d3966eb33cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:13.276857 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c44a2-9a80-4a3a-adf7-0d3966eb33cb admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:13.277131 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c44a2-9a80-4a3a-adf7-0d3966eb33cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:13.277506 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c44a2-9a80-4a3a-adf7-0d3966eb33cb admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:13.278577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c44a2-9a80-4a3a-adf7-0d3966eb33cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:13.281073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c44a2-9a80-4a3a-adf7-0d3966eb33cb admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:13.320993 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 960/1914] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:13 2022] POST /identity/v3/users => generated 301 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:45:13.323405 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222024d-0b09-4e3a-805d-272be8818dbe admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:13.366734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222024d-0b09-4e3a-805d-272be8818dbe admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:13.386867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca426017-c17b-42ec-a60b-c6015e8fe973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:13.387917 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca426017-c17b-42ec-a60b-c6015e8fe973 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:13.388434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca426017-c17b-42ec-a60b-c6015e8fe973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:13.388926 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca426017-c17b-42ec-a60b-c6015e8fe973 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:13.392595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca426017-c17b-42ec-a60b-c6015e8fe973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:13.395857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca426017-c17b-42ec-a60b-c6015e8fe973 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:13.409803 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 955/1915] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:13.420964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c44a2-9a80-4a3a-adf7-0d3966eb33cb admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:13.445343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c44a2-9a80-4a3a-adf7-0d3966eb33cb admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:13.454607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1c925-9130-487e-9d34-c680918e72b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:13.455733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1c925-9130-487e-9d34-c680918e72b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:13.456385 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1c925-9130-487e-9d34-c680918e72b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:13.456964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1c925-9130-487e-9d34-c680918e72b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:13.458397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c1c925-9130-487e-9d34-c680918e72b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:13.461516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c1c925-9130-487e-9d34-c680918e72b6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:13.483889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 961/1916] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:13.489394 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca426017-c17b-42ec-a60b-c6015e8fe973 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:13.511918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca426017-c17b-42ec-a60b-c6015e8fe973 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:13.518902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699ebed-0dc0-4164-9bf9-ce2870d41ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:13.520363 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699ebed-0dc0-4164-9bf9-ce2870d41ceb admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:13.520664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699ebed-0dc0-4164-9bf9-ce2870d41ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:13.521299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699ebed-0dc0-4164-9bf9-ce2870d41ceb admin admin] PATH_INFO: `/v3/projects/952be476569e4bf9abd7cf2cb315cb9a/users/c657f91b5fbf4d9b9c79c560c8c3e3e4/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:13.553051 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699ebed-0dc0-4164-9bf9-ce2870d41ceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=952be476569e4bf9abd7cf2cb315cb9a, user_id=c657f91b5fbf4d9b9c79c560c8c3e3e4, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:13.558067 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699ebed-0dc0-4164-9bf9-ce2870d41ceb admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:13.602328 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 956/1917] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:13 2022] PUT /identity/v3/projects/952be476569e4bf9abd7cf2cb315cb9a/users/c657f91b5fbf4d9b9c79c560c8c3e3e4/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:13.623415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98866813-4f2b-485f-87ab-3ead35385bb0 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:13.624448 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98866813-4f2b-485f-87ab-3ead35385bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:13.624785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98866813-4f2b-485f-87ab-3ead35385bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:13.653167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699ebed-0dc0-4164-9bf9-ce2870d41ceb admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:13.658146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a699ebed-0dc0-4164-9bf9-ce2870d41ceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:13.735961 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-98866813-4f2b-485f-87ab-3ead35385bb0 None None] MFA Rules not processed for user `c657f91b5fbf4d9b9c79c560c8c3e3e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:13.738623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-98866813-4f2b-485f-87ab-3ead35385bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:13.804629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699ebed-0dc0-4164-9bf9-ce2870d41ceb admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:13.810130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81492cf7-da55-48c9-87cf-bbda6429fb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:13.811237 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81492cf7-da55-48c9-87cf-bbda6429fb2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:13.812124 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81492cf7-da55-48c9-87cf-bbda6429fb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:13.812469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81492cf7-da55-48c9-87cf-bbda6429fb2e admin admin] PATH_INFO: `/v3/projects/e4e68229966146958100b8903057e63e` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:13.814881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81492cf7-da55-48c9-87cf-bbda6429fb2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4e68229966146958100b8903057e63e)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:13.816460 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81492cf7-da55-48c9-87cf-bbda6429fb2e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:13.892946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:13.927459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-81492cf7-da55-48c9-87cf-bbda6429fb2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4e68229966146958100b8903057e63e', 'request_id': 'req-81492cf7-da55-48c9-87cf-bbda6429fb2e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:13.937939 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 962/1918] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:13 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:13.938965 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98866813-4f2b-485f-87ab-3ead35385bb0 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:13.942029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-98866813-4f2b-485f-87ab-3ead35385bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:13.945785 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-81492cf7-da55-48c9-87cf-bbda6429fb2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4e68229966146958100b8903057e63e', 'request_id': 'req-81492cf7-da55-48c9-87cf-bbda6429fb2e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:13.962400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-81492cf7-da55-48c9-87cf-bbda6429fb2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4e68229966146958100b8903057e63e', 'request_id': 'req-81492cf7-da55-48c9-87cf-bbda6429fb2e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:13.965573 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98866813-4f2b-485f-87ab-3ead35385bb0 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:13.986063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-81492cf7-da55-48c9-87cf-bbda6429fb2e admin admin] The token cache is being invalidate because project e4e68229966146958100b8903057e63e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:13.986690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-81492cf7-da55-48c9-87cf-bbda6429fb2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:13.988108 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-81492cf7-da55-48c9-87cf-bbda6429fb2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:13.992169 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 957/1919] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:13 2022] DELETE /identity/v3/projects/e4e68229966146958100b8903057e63e => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:45:13.995754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81492cf7-da55-48c9-87cf-bbda6429fb2e admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:13.999548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-81492cf7-da55-48c9-87cf-bbda6429fb2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:14.061125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbf331-ad97-41df-92c1-59652636ab6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:14.062935 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbf331-ad97-41df-92c1-59652636ab6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.062935 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbf331-ad97-41df-92c1-59652636ab6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.062935 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbf331-ad97-41df-92c1-59652636ab6b admin admin] PATH_INFO: `/v3/users/a5fa1023a13f4c0aad4a83d0a53a25d9` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.065145 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbf331-ad97-41df-92c1-59652636ab6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5fa1023a13f4c0aad4a83d0a53a25d9)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:14.068216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbf331-ad97-41df-92c1-59652636ab6b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:14.136459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:14.140173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f2dbf331-ad97-41df-92c1-59652636ab6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5fa1023a13f4c0aad4a83d0a53a25d9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:14.142081 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81492cf7-da55-48c9-87cf-bbda6429fb2e admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:14.149710 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f2dbf331-ad97-41df-92c1-59652636ab6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5fa1023a13f4c0aad4a83d0a53a25d9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:14.150395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd6850-a23e-4767-b60e-57a29831e9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:14.151448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd6850-a23e-4767-b60e-57a29831e9c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.151945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd6850-a23e-4767-b60e-57a29831e9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.152469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd6850-a23e-4767-b60e-57a29831e9c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.153744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd6850-a23e-4767-b60e-57a29831e9c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:14.157572 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd6850-a23e-4767-b60e-57a29831e9c2 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:14.164187 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f2dbf331-ad97-41df-92c1-59652636ab6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5fa1023a13f4c0aad4a83d0a53a25d9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:14.165178 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f2dbf331-ad97-41df-92c1-59652636ab6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5fa1023a13f4c0aad4a83d0a53a25d9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:14.182828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f2dbf331-ad97-41df-92c1-59652636ab6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5fa1023a13f4c0aad4a83d0a53a25d9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:14.195915 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 963/1920] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:13 2022] DELETE /identity/v3/users/a5fa1023a13f4c0aad4a83d0a53a25d9 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:14.198346 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 958/1921] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:13 2022] POST /identity/v3/projects => generated 401 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:14.199258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139928b-0b2f-4ee5-a05d-faf42622696a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.199818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139928b-0b2f-4ee5-a05d-faf42622696a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.200356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139928b-0b2f-4ee5-a05d-faf42622696a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.201861 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45a32e-56f7-4ad2-a7e3-4e88da9ab6dd None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.202406 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45a32e-56f7-4ad2-a7e3-4e88da9ab6dd None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.202897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45a32e-56f7-4ad2-a7e3-4e88da9ab6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.226602 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-0f45a32e-56f7-4ad2-a7e3-4e88da9ab6dd None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:14.228038 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0f45a32e-56f7-4ad2-a7e3-4e88da9ab6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:14.241201 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7139928b-0b2f-4ee5-a05d-faf42622696a None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:14.243223 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7139928b-0b2f-4ee5-a05d-faf42622696a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:14.314886 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 959/1922] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:14.315372 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45a32e-56f7-4ad2-a7e3-4e88da9ab6dd None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:14.316850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0f45a32e-56f7-4ad2-a7e3-4e88da9ab6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:14.331351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45a32e-56f7-4ad2-a7e3-4e88da9ab6dd None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:14.336961 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba4353-26ba-42ed-b503-ac11c12a037c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:14.337755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba4353-26ba-42ed-b503-ac11c12a037c admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.338261 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba4353-26ba-42ed-b503-ac11c12a037c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.338597 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba4353-26ba-42ed-b503-ac11c12a037c admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.339335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba4353-26ba-42ed-b503-ac11c12a037c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:14.340189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 964/1923] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:14.341611 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba4353-26ba-42ed-b503-ac11c12a037c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:14.342938 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139928b-0b2f-4ee5-a05d-faf42622696a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:14.347698 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7139928b-0b2f-4ee5-a05d-faf42622696a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:14.370088 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 960/1924] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:14 2022] POST /identity/v3/users => generated 307 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:14.371400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad4602-d5d9-4e9a-b5ed-044836b7d380 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.371757 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad4602-d5d9-4e9a-b5ed-044836b7d380 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.372151 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad4602-d5d9-4e9a-b5ed-044836b7d380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.393845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a5ad4602-d5d9-4e9a-b5ed-044836b7d380 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:14.395351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a5ad4602-d5d9-4e9a-b5ed-044836b7d380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:14.406683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 961/1925] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:14.406683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad4602-d5d9-4e9a-b5ed-044836b7d380 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:14.421161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad4602-d5d9-4e9a-b5ed-044836b7d380 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:14.426745 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bfc9a-db21-4aab-9bbb-31b153170d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:14.427504 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bfc9a-db21-4aab-9bbb-31b153170d85 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.427644 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bfc9a-db21-4aab-9bbb-31b153170d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.427923 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bfc9a-db21-4aab-9bbb-31b153170d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.428620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bfc9a-db21-4aab-9bbb-31b153170d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:14.430826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bfc9a-db21-4aab-9bbb-31b153170d85 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:14.446567 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 962/1926] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:14.446567 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bfc9a-db21-4aab-9bbb-31b153170d85 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:14.449180 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139928b-0b2f-4ee5-a05d-faf42622696a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:14.454668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bfc9a-db21-4aab-9bbb-31b153170d85 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:14.455677 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3f8eb-14af-4e4f-b91b-022a2f174b09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:14.456785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3f8eb-14af-4e4f-b91b-022a2f174b09 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.457276 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3f8eb-14af-4e4f-b91b-022a2f174b09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.457680 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3f8eb-14af-4e4f-b91b-022a2f174b09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.459973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121ff16-1903-43f8-992f-8b16359b164a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:14.460854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121ff16-1903-43f8-992f-8b16359b164a admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.460965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121ff16-1903-43f8-992f-8b16359b164a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.461228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121ff16-1903-43f8-992f-8b16359b164a admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.461919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121ff16-1903-43f8-992f-8b16359b164a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:45:14.462188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121ff16-1903-43f8-992f-8b16359b164a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:14.463882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121ff16-1903-43f8-992f-8b16359b164a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:14.474972 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3f8eb-14af-4e4f-b91b-022a2f174b09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:14.477952 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 963/1927] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:45:14.478131 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3f8eb-14af-4e4f-b91b-022a2f174b09 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:14.481112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121ff16-1903-43f8-992f-8b16359b164a admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:14.488513 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:14.493634 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121ff16-1903-43f8-992f-8b16359b164a admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:14.497587 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 965/1928] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:14.499065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3f8eb-14af-4e4f-b91b-022a2f174b09 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:14.501517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b1e71-23e6-4ed0-be31-502e39ad965d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:14.502353 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b1e71-23e6-4ed0-be31-502e39ad965d admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.502816 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b1e71-23e6-4ed0-be31-502e39ad965d admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.503280 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b1e71-23e6-4ed0-be31-502e39ad965d admin admin] PATH_INFO: `/v3/projects/b45627ad48a14d4d9b2ca63774042e13/users/25ed9f03e11140a280a2f107fc9aafa2/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.516786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3f8eb-14af-4e4f-b91b-022a2f174b09 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:14.525650 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105ab07-4112-4c60-ae64-60d859c8ebc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:14.526769 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105ab07-4112-4c60-ae64-60d859c8ebc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.527256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105ab07-4112-4c60-ae64-60d859c8ebc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.527749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105ab07-4112-4c60-ae64-60d859c8ebc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.530446 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105ab07-4112-4c60-ae64-60d859c8ebc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:14.532951 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105ab07-4112-4c60-ae64-60d859c8ebc7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:14.534663 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b1e71-23e6-4ed0-be31-502e39ad965d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b45627ad48a14d4d9b2ca63774042e13, user_id=25ed9f03e11140a280a2f107fc9aafa2, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:14.537525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b1e71-23e6-4ed0-be31-502e39ad965d admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:14.563951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 964/1929] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:14 2022] PUT /identity/v3/projects/b45627ad48a14d4d9b2ca63774042e13/users/25ed9f03e11140a280a2f107fc9aafa2/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:14.566111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b1e71-23e6-4ed0-be31-502e39ad965d admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:14.589227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 966/1930] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:14 2022] POST /identity/v3/projects => generated 361 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:14.594793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105ab07-4112-4c60-ae64-60d859c8ebc7 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:14.657069 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b1e71-23e6-4ed0-be31-502e39ad965d admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:14.671414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:14.699605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105ab07-4112-4c60-ae64-60d859c8ebc7 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:14.711835 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b38f3e-d66f-4cea-beda-e4da5d9d5063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:14.713010 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b38f3e-d66f-4cea-beda-e4da5d9d5063 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.714651 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b38f3e-d66f-4cea-beda-e4da5d9d5063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.715115 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b38f3e-d66f-4cea-beda-e4da5d9d5063 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.717431 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b38f3e-d66f-4cea-beda-e4da5d9d5063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:14.722921 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b38f3e-d66f-4cea-beda-e4da5d9d5063 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:14.750441 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239f805-5876-4ac9-b7c4-eaf886314453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:14.751745 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239f805-5876-4ac9-b7c4-eaf886314453 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.752120 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239f805-5876-4ac9-b7c4-eaf886314453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.752597 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239f805-5876-4ac9-b7c4-eaf886314453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.753884 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239f805-5876-4ac9-b7c4-eaf886314453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:14.757395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239f805-5876-4ac9-b7c4-eaf886314453 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:14.766261 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 967/1931] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:14 2022] POST /identity/v3/users => generated 281 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:14.771848 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b38f3e-d66f-4cea-beda-e4da5d9d5063 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:14.780244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 965/1932] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:14.786315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239f805-5876-4ac9-b7c4-eaf886314453 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:14.802623 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239f805-5876-4ac9-b7c4-eaf886314453 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:14.806639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b38f3e-d66f-4cea-beda-e4da5d9d5063 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:14.807955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad7e23-4af0-442f-8695-4b9d85e23997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:14.808686 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad7e23-4af0-442f-8695-4b9d85e23997 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.808969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad7e23-4af0-442f-8695-4b9d85e23997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.809242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad7e23-4af0-442f-8695-4b9d85e23997 admin admin] PATH_INFO: `/v3/projects/b45627ad48a14d4d9b2ca63774042e13/users/25ed9f03e11140a280a2f107fc9aafa2/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.812082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad7e23-4af0-442f-8695-4b9d85e23997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b45627ad48a14d4d9b2ca63774042e13, user_id=25ed9f03e11140a280a2f107fc9aafa2, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:14.815125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad7e23-4af0-442f-8695-4b9d85e23997 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:14.820184 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d336d6-d59f-4c80-b5d8-baa88143fdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:14.821115 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d336d6-d59f-4c80-b5d8-baa88143fdec admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.821568 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d336d6-d59f-4c80-b5d8-baa88143fdec admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.822045 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d336d6-d59f-4c80-b5d8-baa88143fdec admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.822906 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d336d6-d59f-4c80-b5d8-baa88143fdec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:14.824818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d336d6-d59f-4c80-b5d8-baa88143fdec admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:14.837453 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 966/1933] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:14 2022] PUT /identity/v3/projects/b45627ad48a14d4d9b2ca63774042e13/users/25ed9f03e11140a280a2f107fc9aafa2/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:14.839301 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 968/1934] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:14.841499 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad7e23-4af0-442f-8695-4b9d85e23997 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:14.843663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfd769-b814-4f46-bd50-10e89727f04b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.843938 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfd769-b814-4f46-bd50-10e89727f04b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.844240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfd769-b814-4f46-bd50-10e89727f04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.889036 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-6abfd769-b814-4f46-bd50-10e89727f04b None None] MFA Rules not processed for user `25ed9f03e11140a280a2f107fc9aafa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:14.894175 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6abfd769-b814-4f46-bd50-10e89727f04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:14.964915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad7e23-4af0-442f-8695-4b9d85e23997 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:14.971320 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50db9c9-aad4-4aa5-b01a-74d3c2093177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:14.972710 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50db9c9-aad4-4aa5-b01a-74d3c2093177 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.972992 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50db9c9-aad4-4aa5-b01a-74d3c2093177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.973323 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50db9c9-aad4-4aa5-b01a-74d3c2093177 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.974489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50db9c9-aad4-4aa5-b01a-74d3c2093177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:14.976197 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50db9c9-aad4-4aa5-b01a-74d3c2093177 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:14.997489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 967/1935] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:15.004731 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50db9c9-aad4-4aa5-b01a-74d3c2093177 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:15.021752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50db9c9-aad4-4aa5-b01a-74d3c2093177 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:15.030152 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98acd3-3468-404e-8238-d9d230faad9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:15.032352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98acd3-3468-404e-8238-d9d230faad9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:15.032964 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98acd3-3468-404e-8238-d9d230faad9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:15.033779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98acd3-3468-404e-8238-d9d230faad9d admin admin] PATH_INFO: `/v3/projects/eb78be7e3a44481d92e7b06718d75815/users/737182a8cc1547db8b8028added9c0e3/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:15.038214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:15.059804 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 969/1936] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:15.061383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfd769-b814-4f46-bd50-10e89727f04b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:15.064141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6abfd769-b814-4f46-bd50-10e89727f04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:15.086061 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98acd3-3468-404e-8238-d9d230faad9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb78be7e3a44481d92e7b06718d75815, user_id=737182a8cc1547db8b8028added9c0e3, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:15.091740 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98acd3-3468-404e-8238-d9d230faad9d admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:15.112356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfd769-b814-4f46-bd50-10e89727f04b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:15.125508 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 968/1937] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:15 2022] PUT /identity/v3/projects/eb78be7e3a44481d92e7b06718d75815/users/737182a8cc1547db8b8028added9c0e3/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:15.127102 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98acd3-3468-404e-8238-d9d230faad9d admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:15.130236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4b98acd3-3468-404e-8238-d9d230faad9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:15.259690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748aafd-9158-4f5f-9468-2e48af6f7d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:15.260784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748aafd-9158-4f5f-9468-2e48af6f7d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:15.261278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748aafd-9158-4f5f-9468-2e48af6f7d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:15.261819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748aafd-9158-4f5f-9468-2e48af6f7d79 admin admin] PATH_INFO: `/v3/projects/bc5ac77baea4479facf1af48f5b7dbf1` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:15.263404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748aafd-9158-4f5f-9468-2e48af6f7d79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc5ac77baea4479facf1af48f5b7dbf1)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:15.265577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748aafd-9158-4f5f-9468-2e48af6f7d79 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:15.295878 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98acd3-3468-404e-8238-d9d230faad9d admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:15.300827 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de158e04-ca32-4333-9167-4f122076f50a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:15.301350 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158e04-ca32-4333-9167-4f122076f50a service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:15.301832 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158e04-ca32-4333-9167-4f122076f50a service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:15.302245 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158e04-ca32-4333-9167-4f122076f50a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:15.371100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:15.374658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5748aafd-9158-4f5f-9468-2e48af6f7d79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc5ac77baea4479facf1af48f5b7dbf1', 'request_id': 'req-5748aafd-9158-4f5f-9468-2e48af6f7d79'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:15.388707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de158e04-ca32-4333-9167-4f122076f50a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:15.390507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de158e04-ca32-4333-9167-4f122076f50a service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:15.397211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5748aafd-9158-4f5f-9468-2e48af6f7d79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc5ac77baea4479facf1af48f5b7dbf1', 'request_id': 'req-5748aafd-9158-4f5f-9468-2e48af6f7d79'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:15.416209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5748aafd-9158-4f5f-9468-2e48af6f7d79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc5ac77baea4479facf1af48f5b7dbf1', 'request_id': 'req-5748aafd-9158-4f5f-9468-2e48af6f7d79'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:15.416369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 969/1938] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:15 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:15.417977 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f265e-f3f6-401d-931f-e950a5ee98de None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:15.418268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f265e-f3f6-401d-931f-e950a5ee98de None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:15.418542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f265e-f3f6-401d-931f-e950a5ee98de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:15.447661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5748aafd-9158-4f5f-9468-2e48af6f7d79 admin admin] The token cache is being invalidate because project bc5ac77baea4479facf1af48f5b7dbf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:15.447974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5748aafd-9158-4f5f-9468-2e48af6f7d79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:15.448891 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5748aafd-9158-4f5f-9468-2e48af6f7d79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:15.454832 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 970/1939] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:15 2022] DELETE /identity/v3/projects/bc5ac77baea4479facf1af48f5b7dbf1 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:15.455883 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748aafd-9158-4f5f-9468-2e48af6f7d79 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:15.457438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5748aafd-9158-4f5f-9468-2e48af6f7d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:15.491716 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-806f265e-f3f6-401d-931f-e950a5ee98de None None] MFA Rules not processed for user `737182a8cc1547db8b8028added9c0e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:15.492307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-806f265e-f3f6-401d-931f-e950a5ee98de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:15.575189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748aafd-9158-4f5f-9468-2e48af6f7d79 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:15.594872 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc3069-e465-4ed8-8066-e824c348bb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:15.596201 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc3069-e465-4ed8-8066-e824c348bb3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:15.596705 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc3069-e465-4ed8-8066-e824c348bb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:15.597384 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc3069-e465-4ed8-8066-e824c348bb3e admin admin] PATH_INFO: `/v3/users/0321d1017e1d4672851f0693491798a9` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:15.598833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc3069-e465-4ed8-8066-e824c348bb3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0321d1017e1d4672851f0693491798a9)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:15.601172 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc3069-e465-4ed8-8066-e824c348bb3e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:15.670076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bcdc3069-e465-4ed8-8066-e824c348bb3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0321d1017e1d4672851f0693491798a9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:15.670449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 970/1940] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:15.672188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f265e-f3f6-401d-931f-e950a5ee98de None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:15.674544 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-806f265e-f3f6-401d-931f-e950a5ee98de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:15.686873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bcdc3069-e465-4ed8-8066-e824c348bb3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0321d1017e1d4672851f0693491798a9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:15.700975 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bcdc3069-e465-4ed8-8066-e824c348bb3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0321d1017e1d4672851f0693491798a9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:15.701892 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bcdc3069-e465-4ed8-8066-e824c348bb3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0321d1017e1d4672851f0693491798a9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:15.719382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bcdc3069-e465-4ed8-8066-e824c348bb3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0321d1017e1d4672851f0693491798a9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:15.727002 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 971/1941] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:15 2022] DELETE /identity/v3/users/0321d1017e1d4672851f0693491798a9 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:15.727989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc3069-e465-4ed8-8066-e824c348bb3e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:15.729687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bcdc3069-e465-4ed8-8066-e824c348bb3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:15.730385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:15.796159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f265e-f3f6-401d-931f-e950a5ee98de None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:15.815813 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc3069-e465-4ed8-8066-e824c348bb3e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:15.820620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b2251-3946-4356-84e4-69b4c1eb23cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:15.821743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b2251-3946-4356-84e4-69b4c1eb23cd service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:15.821743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b2251-3946-4356-84e4-69b4c1eb23cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:15.822010 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b2251-3946-4356-84e4-69b4c1eb23cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:15.824030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-979b2251-3946-4356-84e4-69b4c1eb23cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:15.836561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:15.891038 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e36f77-0761-4a0f-8887-23aaf34f8d7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:15.892198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e36f77-0761-4a0f-8887-23aaf34f8d7c service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:15.893163 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e36f77-0761-4a0f-8887-23aaf34f8d7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:15.893163 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e36f77-0761-4a0f-8887-23aaf34f8d7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:15.896357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b7e36f77-0761-4a0f-8887-23aaf34f8d7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:15.942196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b2251-3946-4356-84e4-69b4c1eb23cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:15.945264 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b2251-3946-4356-84e4-69b4c1eb23cd service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:15.964994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 972/1942] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:15.990912 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b2251-3946-4356-84e4-69b4c1eb23cd service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:15.993228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-979b2251-3946-4356-84e4-69b4c1eb23cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:16.053040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e36f77-0761-4a0f-8887-23aaf34f8d7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:16.053040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e36f77-0761-4a0f-8887-23aaf34f8d7c service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:16.082017 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 971/1943] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:15 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:16.141110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b2251-3946-4356-84e4-69b4c1eb23cd service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:16.151322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee297e52-4709-4690-836d-a1440b1b134b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:16.152425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297e52-4709-4690-836d-a1440b1b134b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:16.152953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297e52-4709-4690-836d-a1440b1b134b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:16.153349 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297e52-4709-4690-836d-a1440b1b134b admin admin] PATH_INFO: `/v3/users/737182a8cc1547db8b8028added9c0e3` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:16.155765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee297e52-4709-4690-836d-a1440b1b134b admin admin] RBAC: Authorizing `identity:delete_user(user_id=737182a8cc1547db8b8028added9c0e3)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:16.161743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee297e52-4709-4690-836d-a1440b1b134b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:16.231510 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e36f77-0761-4a0f-8887-23aaf34f8d7c service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:16.235292 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b7e36f77-0761-4a0f-8887-23aaf34f8d7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:16.297724 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e36f77-0761-4a0f-8887-23aaf34f8d7c service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:16.307648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58873557-3d29-4a5b-a330-4f233d90f94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:16.308481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58873557-3d29-4a5b-a330-4f233d90f94f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:16.309004 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58873557-3d29-4a5b-a330-4f233d90f94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:16.309814 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58873557-3d29-4a5b-a330-4f233d90f94f admin admin] PATH_INFO: `/v3/projects/d8f40e0afaa24f40bb7c4112864a7bf3` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:16.323903 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58873557-3d29-4a5b-a330-4f233d90f94f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8f40e0afaa24f40bb7c4112864a7bf3)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:16.330109 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58873557-3d29-4a5b-a330-4f233d90f94f admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:16.368870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ee297e52-4709-4690-836d-a1440b1b134b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '737182a8cc1547db8b8028added9c0e3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:16.390618 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ee297e52-4709-4690-836d-a1440b1b134b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '737182a8cc1547db8b8028added9c0e3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:16.410936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ee297e52-4709-4690-836d-a1440b1b134b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '737182a8cc1547db8b8028added9c0e3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:16.411567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ee297e52-4709-4690-836d-a1440b1b134b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '737182a8cc1547db8b8028added9c0e3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:16.431854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ee297e52-4709-4690-836d-a1440b1b134b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '737182a8cc1547db8b8028added9c0e3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:16.445382 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-58873557-3d29-4a5b-a330-4f233d90f94f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8f40e0afaa24f40bb7c4112864a7bf3', 'request_id': 'req-58873557-3d29-4a5b-a330-4f233d90f94f'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:16.453483 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-58873557-3d29-4a5b-a330-4f233d90f94f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8f40e0afaa24f40bb7c4112864a7bf3', 'request_id': 'req-58873557-3d29-4a5b-a330-4f233d90f94f'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:16.460775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 973/1944] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:15 2022] DELETE /identity/v3/users/737182a8cc1547db8b8028added9c0e3 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:16.472581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-58873557-3d29-4a5b-a330-4f233d90f94f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8f40e0afaa24f40bb7c4112864a7bf3', 'request_id': 'req-58873557-3d29-4a5b-a330-4f233d90f94f'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:16.474426 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290f11e-e3b3-42a5-84b7-1063c4aa1ae7 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:16.474901 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290f11e-e3b3-42a5-84b7-1063c4aa1ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:16.475400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290f11e-e3b3-42a5-84b7-1063c4aa1ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:16.512314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-58873557-3d29-4a5b-a330-4f233d90f94f admin admin] The token cache is being invalidate because project d8f40e0afaa24f40bb7c4112864a7bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:16.512596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-58873557-3d29-4a5b-a330-4f233d90f94f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:16.517477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-3290f11e-e3b3-42a5-84b7-1063c4aa1ae7 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:16.517779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-58873557-3d29-4a5b-a330-4f233d90f94f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:16.519676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3290f11e-e3b3-42a5-84b7-1063c4aa1ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:16.525427 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 972/1945] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:16 2022] DELETE /identity/v3/projects/d8f40e0afaa24f40bb7c4112864a7bf3 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:16.531033 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a6acd-267a-4acb-a691-73e6925fa85d None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:16.531657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a6acd-267a-4acb-a691-73e6925fa85d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:16.531807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a6acd-267a-4acb-a691-73e6925fa85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:16.559538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:16.617278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-ec9a6acd-267a-4acb-a691-73e6925fa85d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:16.618631 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-ec9a6acd-267a-4acb-a691-73e6925fa85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:16.637884 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 974/1946] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:16.648262 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 973/1947] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:16.661764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290f11e-e3b3-42a5-84b7-1063c4aa1ae7 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:16.663345 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3290f11e-e3b3-42a5-84b7-1063c4aa1ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:16.666297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0a49b-d49e-4a8b-9fe2-6b1c68f203f2 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:16.666582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0a49b-d49e-4a8b-9fe2-6b1c68f203f2 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:16.668196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0a49b-d49e-4a8b-9fe2-6b1c68f203f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:16.742520 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-e8b0a49b-d49e-4a8b-9fe2-6b1c68f203f2 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:16.743820 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e8b0a49b-d49e-4a8b-9fe2-6b1c68f203f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:16.776308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 974/1948] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:16.788969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0a49b-d49e-4a8b-9fe2-6b1c68f203f2 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:16.793627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:16.815587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0a49b-d49e-4a8b-9fe2-6b1c68f203f2 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:16.821835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaf017-857d-4f3e-9059-a21647c2fd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:16.822421 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaf017-857d-4f3e-9059-a21647c2fd67 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:16.822703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaf017-857d-4f3e-9059-a21647c2fd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:16.823085 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaf017-857d-4f3e-9059-a21647c2fd67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:16.824111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aaf017-857d-4f3e-9059-a21647c2fd67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:45:16.824287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aaf017-857d-4f3e-9059-a21647c2fd67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:16.826252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aaf017-857d-4f3e-9059-a21647c2fd67 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:16.847926 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290f11e-e3b3-42a5-84b7-1063c4aa1ae7 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:16.853958 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412c574-79a6-4382-9076-9e264c328a15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:16.855460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412c574-79a6-4382-9076-9e264c328a15 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:16.855956 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412c574-79a6-4382-9076-9e264c328a15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:16.859727 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412c574-79a6-4382-9076-9e264c328a15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:16.877228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 975/1949] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:45:16.881960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaf017-857d-4f3e-9059-a21647c2fd67 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:16.894084 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaf017-857d-4f3e-9059-a21647c2fd67 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:16.899301 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d7eae-191d-4816-b1a1-6e031f7fab40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:16.899915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d7eae-191d-4816-b1a1-6e031f7fab40 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:16.900452 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d7eae-191d-4816-b1a1-6e031f7fab40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:16.900780 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d7eae-191d-4816-b1a1-6e031f7fab40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:16.901510 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d7eae-191d-4816-b1a1-6e031f7fab40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:16.903559 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d7eae-191d-4816-b1a1-6e031f7fab40 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:16.903972 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412c574-79a6-4382-9076-9e264c328a15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:16.909482 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412c574-79a6-4382-9076-9e264c328a15 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:16.924967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 975/1950] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:16.947687 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 976/1951] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:16 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:16.948971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412c574-79a6-4382-9076-9e264c328a15 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:16.969364 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412c574-79a6-4382-9076-9e264c328a15 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:17.052672 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084005a6-7dd5-45d7-b6e8-092015fce2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:17.053424 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084005a6-7dd5-45d7-b6e8-092015fce2d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:17.053747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084005a6-7dd5-45d7-b6e8-092015fce2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:17.054179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084005a6-7dd5-45d7-b6e8-092015fce2d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:17.054840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084005a6-7dd5-45d7-b6e8-092015fce2d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:17.057765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084005a6-7dd5-45d7-b6e8-092015fce2d1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:17.095205 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 976/1952] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:16 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:17.099976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d7eae-191d-4816-b1a1-6e031f7fab40 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:17.115481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d7eae-191d-4816-b1a1-6e031f7fab40 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:17.122731 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731aeab8-c3e7-4fa9-b3b2-35135a968f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:17.123566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731aeab8-c3e7-4fa9-b3b2-35135a968f0b admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:17.123849 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731aeab8-c3e7-4fa9-b3b2-35135a968f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:17.124158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731aeab8-c3e7-4fa9-b3b2-35135a968f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:17.126034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731aeab8-c3e7-4fa9-b3b2-35135a968f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:17.128871 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731aeab8-c3e7-4fa9-b3b2-35135a968f0b admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:17.148775 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 977/1953] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:17.151177 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084005a6-7dd5-45d7-b6e8-092015fce2d1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:17.175017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084005a6-7dd5-45d7-b6e8-092015fce2d1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:17.184883 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1adfd-6a18-437d-8533-89b02984f826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:17.185964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1adfd-6a18-437d-8533-89b02984f826 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:17.186802 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1adfd-6a18-437d-8533-89b02984f826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:17.187312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1adfd-6a18-437d-8533-89b02984f826 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:17.188870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1adfd-6a18-437d-8533-89b02984f826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:17.191907 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1adfd-6a18-437d-8533-89b02984f826 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:17.220912 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 977/1954] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:17.222666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731aeab8-c3e7-4fa9-b3b2-35135a968f0b admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:17.237438 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731aeab8-c3e7-4fa9-b3b2-35135a968f0b admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:17.264233 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a2941-5fdd-474f-b263-fbd6138b614a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:17.265154 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a2941-5fdd-474f-b263-fbd6138b614a admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:17.265505 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a2941-5fdd-474f-b263-fbd6138b614a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:17.265766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a2941-5fdd-474f-b263-fbd6138b614a admin admin] PATH_INFO: `/v3/projects/1a7e365ff2ed4dad860ef2388da58cdc/users/3223d157076a42689adf85c68228bda4/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:17.298498 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a2941-5fdd-474f-b263-fbd6138b614a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a7e365ff2ed4dad860ef2388da58cdc, user_id=3223d157076a42689adf85c68228bda4, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:17.302896 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a2941-5fdd-474f-b263-fbd6138b614a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:17.336554 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 978/1955] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:17 2022] PUT /identity/v3/projects/1a7e365ff2ed4dad860ef2388da58cdc/users/3223d157076a42689adf85c68228bda4/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:45:17.348036 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f065d0-347c-4212-a016-9587e7298496 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:17.348369 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f065d0-347c-4212-a016-9587e7298496 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:17.349416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f065d0-347c-4212-a016-9587e7298496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:17.398107 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c7f065d0-347c-4212-a016-9587e7298496 None None] MFA Rules not processed for user `3223d157076a42689adf85c68228bda4`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:17.400571 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c7f065d0-347c-4212-a016-9587e7298496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:17.481269 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:17.507515 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a2941-5fdd-474f-b263-fbd6138b614a admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:17.509777 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d45a2941-5fdd-474f-b263-fbd6138b614a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:17.593705 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 978/1956] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:17.600438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f065d0-347c-4212-a016-9587e7298496 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:17.633548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a2941-5fdd-474f-b263-fbd6138b614a admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:17.640749 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae7498-2408-478b-8efb-680676b15a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:17.641425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae7498-2408-478b-8efb-680676b15a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:17.641704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae7498-2408-478b-8efb-680676b15a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:17.642164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae7498-2408-478b-8efb-680676b15a80 admin admin] PATH_INFO: `/v3/projects/eb78be7e3a44481d92e7b06718d75815` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:17.644159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae7498-2408-478b-8efb-680676b15a80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb78be7e3a44481d92e7b06718d75815)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:17.646187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae7498-2408-478b-8efb-680676b15a80 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:17.714042 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bfae7498-2408-478b-8efb-680676b15a80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb78be7e3a44481d92e7b06718d75815', 'request_id': 'req-bfae7498-2408-478b-8efb-680676b15a80'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:17.719791 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f065d0-347c-4212-a016-9587e7298496 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:17.721601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bfae7498-2408-478b-8efb-680676b15a80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb78be7e3a44481d92e7b06718d75815', 'request_id': 'req-bfae7498-2408-478b-8efb-680676b15a80'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:17.728710 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bfae7498-2408-478b-8efb-680676b15a80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb78be7e3a44481d92e7b06718d75815', 'request_id': 'req-bfae7498-2408-478b-8efb-680676b15a80'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:17.746341 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bfae7498-2408-478b-8efb-680676b15a80 admin admin] The token cache is being invalidate because project eb78be7e3a44481d92e7b06718d75815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:17.746634 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bfae7498-2408-478b-8efb-680676b15a80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:17.747409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bfae7498-2408-478b-8efb-680676b15a80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:17.749309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 979/1957] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:17 2022] DELETE /identity/v3/projects/eb78be7e3a44481d92e7b06718d75815 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:45:17.754829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21950bf6-25b5-45ec-8d0d-fa387342281d None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:17.755138 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21950bf6-25b5-45ec-8d0d-fa387342281d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:17.755340 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21950bf6-25b5-45ec-8d0d-fa387342281d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:17.785916 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-21950bf6-25b5-45ec-8d0d-fa387342281d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:17.787175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-21950bf6-25b5-45ec-8d0d-fa387342281d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:17.825371 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:17.864502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4f4fd-6039-4354-812a-1575497a9b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:17.865678 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4f4fd-6039-4354-812a-1575497a9b46 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:17.866023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4f4fd-6039-4354-812a-1575497a9b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:17.867449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4f4fd-6039-4354-812a-1575497a9b46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:17.868656 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4f4fd-6039-4354-812a-1575497a9b46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:17.872788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4f4fd-6039-4354-812a-1575497a9b46 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:17.958857 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 979/1958] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:17 2022] POST /identity/v3/projects => generated 377 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:45:17.965927 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4f4fd-6039-4354-812a-1575497a9b46 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:17.975808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8ad4f4fd-6039-4354-812a-1575497a9b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:17.988728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 980/1959] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:17.997530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc649a14-ddc8-43eb-81d3-8c2c35356362 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:17.998032 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc649a14-ddc8-43eb-81d3-8c2c35356362 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:17.998425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc649a14-ddc8-43eb-81d3-8c2c35356362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:18.061191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-dc649a14-ddc8-43eb-81d3-8c2c35356362 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:18.063857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-dc649a14-ddc8-43eb-81d3-8c2c35356362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:18.135373 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4f4fd-6039-4354-812a-1575497a9b46 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:18.144107 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5392cf5-b4bb-4c8a-bf11-1bce0d9d23de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:18.145339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5392cf5-b4bb-4c8a-bf11-1bce0d9d23de admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:18.145839 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5392cf5-b4bb-4c8a-bf11-1bce0d9d23de admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:18.146543 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5392cf5-b4bb-4c8a-bf11-1bce0d9d23de admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:18.147453 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5392cf5-b4bb-4c8a-bf11-1bce0d9d23de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:18.149762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5392cf5-b4bb-4c8a-bf11-1bce0d9d23de admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:18.185928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 980/1960] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:17 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:18.191271 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5392cf5-b4bb-4c8a-bf11-1bce0d9d23de admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:18.201198 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:18.214854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5392cf5-b4bb-4c8a-bf11-1bce0d9d23de admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:18.230316 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd3728-c460-4fab-8ba9-2f014bdde3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:18.231632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd3728-c460-4fab-8ba9-2f014bdde3ba admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:18.232331 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd3728-c460-4fab-8ba9-2f014bdde3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:18.233359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd3728-c460-4fab-8ba9-2f014bdde3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:18.235873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd3728-c460-4fab-8ba9-2f014bdde3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:18.244823 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd3728-c460-4fab-8ba9-2f014bdde3ba admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:18.259463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 981/1961] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:18.265871 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc649a14-ddc8-43eb-81d3-8c2c35356362 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:18.292136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc649a14-ddc8-43eb-81d3-8c2c35356362 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:18.300491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 981/1962] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:18.306028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd3728-c460-4fab-8ba9-2f014bdde3ba admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:18.308283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4ae56-d4e6-4187-98e3-fbcaa0ac8e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:18.309460 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4ae56-d4e6-4187-98e3-fbcaa0ac8e7b admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:18.309460 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4ae56-d4e6-4187-98e3-fbcaa0ac8e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:18.309460 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4ae56-d4e6-4187-98e3-fbcaa0ac8e7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:18.310251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba4ae56-d4e6-4187-98e3-fbcaa0ac8e7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:45:18.310505 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba4ae56-d4e6-4187-98e3-fbcaa0ac8e7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:18.313150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba4ae56-d4e6-4187-98e3-fbcaa0ac8e7b admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:18.326582 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd3728-c460-4fab-8ba9-2f014bdde3ba admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:18.350828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 982/1963] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:45:18.357319 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863893d9-6dd6-475e-ae6b-f9158875fb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:18.357580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4ae56-d4e6-4187-98e3-fbcaa0ac8e7b admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:18.358399 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863893d9-6dd6-475e-ae6b-f9158875fb1f admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:18.358879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863893d9-6dd6-475e-ae6b-f9158875fb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:18.359436 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863893d9-6dd6-475e-ae6b-f9158875fb1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:18.360868 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863893d9-6dd6-475e-ae6b-f9158875fb1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:18.364626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863893d9-6dd6-475e-ae6b-f9158875fb1f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:18.396439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 982/1964] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:18.404383 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4ae56-d4e6-4187-98e3-fbcaa0ac8e7b admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:18.406517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863893d9-6dd6-475e-ae6b-f9158875fb1f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:18.415479 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c52a7-c590-4c12-8c95-24d0426080cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:18.416758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c52a7-c590-4c12-8c95-24d0426080cd admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:18.417408 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c52a7-c590-4c12-8c95-24d0426080cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:18.417735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c52a7-c590-4c12-8c95-24d0426080cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:18.419209 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c52a7-c590-4c12-8c95-24d0426080cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:18.424543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c52a7-c590-4c12-8c95-24d0426080cd admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:18.431648 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863893d9-6dd6-475e-ae6b-f9158875fb1f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:18.442076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964522b-3fe0-408f-b8ff-0aaa51c0ab53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:18.443199 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964522b-3fe0-408f-b8ff-0aaa51c0ab53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:18.443665 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964522b-3fe0-408f-b8ff-0aaa51c0ab53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:18.444167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964522b-3fe0-408f-b8ff-0aaa51c0ab53 admin admin] PATH_INFO: `/v3/projects/d0c450b38f424605b47db03f5285fc7c/users/97d86e2016f444288f8e96a4f74cf165/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:18.473826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 983/1965] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:18 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:45:18.475531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964522b-3fe0-408f-b8ff-0aaa51c0ab53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0c450b38f424605b47db03f5285fc7c, user_id=97d86e2016f444288f8e96a4f74cf165, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:18.479652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c52a7-c590-4c12-8c95-24d0426080cd admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:18.481161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964522b-3fe0-408f-b8ff-0aaa51c0ab53 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:18.517704 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 983/1966] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:18 2022] PUT /identity/v3/projects/d0c450b38f424605b47db03f5285fc7c/users/97d86e2016f444288f8e96a4f74cf165/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:18.537409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fb9ce-bb9b-49d6-9f17-4cf0cb33e121 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:18.537849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fb9ce-bb9b-49d6-9f17-4cf0cb33e121 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:18.538299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fb9ce-bb9b-49d6-9f17-4cf0cb33e121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:18.660919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a15fb9ce-bb9b-49d6-9f17-4cf0cb33e121 None None] MFA Rules not processed for user `97d86e2016f444288f8e96a4f74cf165`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:18.661532 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c52a7-c590-4c12-8c95-24d0426080cd admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:18.669045 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a15fb9ce-bb9b-49d6-9f17-4cf0cb33e121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:18.722584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:18.777273 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cb5ad-b032-4ee2-9f07-256afca65725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:18.777931 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cb5ad-b032-4ee2-9f07-256afca65725 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:18.778207 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cb5ad-b032-4ee2-9f07-256afca65725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:18.778481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cb5ad-b032-4ee2-9f07-256afca65725 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:18.780354 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cb5ad-b032-4ee2-9f07-256afca65725 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:18.782829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cb5ad-b032-4ee2-9f07-256afca65725 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:18.818150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 984/1967] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:18 2022] POST /identity/v3/users => generated 288 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:18.819724 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cb5ad-b032-4ee2-9f07-256afca65725 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:18.835292 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cb5ad-b032-4ee2-9f07-256afca65725 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:18.845973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a67064-e2c1-4a53-b552-941d5eb9ed0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:18.847936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a67064-e2c1-4a53-b552-941d5eb9ed0e admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:18.847936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a67064-e2c1-4a53-b552-941d5eb9ed0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:18.848353 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a67064-e2c1-4a53-b552-941d5eb9ed0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:18.849383 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a67064-e2c1-4a53-b552-941d5eb9ed0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:18.852835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a67064-e2c1-4a53-b552-941d5eb9ed0e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:18.859567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 984/1968] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:18.865856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fb9ce-bb9b-49d6-9f17-4cf0cb33e121 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:18.873846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 985/1969] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:18.880256 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a67064-e2c1-4a53-b552-941d5eb9ed0e admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:18.894079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a67064-e2c1-4a53-b552-941d5eb9ed0e admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:18.899798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fb9ce-bb9b-49d6-9f17-4cf0cb33e121 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:18.901770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b60ed6-d0ed-4bc9-a730-66e0e595a7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:18.902471 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b60ed6-d0ed-4bc9-a730-66e0e595a7de admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:18.903014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b60ed6-d0ed-4bc9-a730-66e0e595a7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:18.903403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b60ed6-d0ed-4bc9-a730-66e0e595a7de admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:18.904367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b60ed6-d0ed-4bc9-a730-66e0e595a7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:18.906037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b60ed6-d0ed-4bc9-a730-66e0e595a7de admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:18.907667 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c95816-c63f-4e69-a329-197357242148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:18.909283 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c95816-c63f-4e69-a329-197357242148 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:18.909828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c95816-c63f-4e69-a329-197357242148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:18.910925 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c95816-c63f-4e69-a329-197357242148 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:18.912294 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c95816-c63f-4e69-a329-197357242148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:18.916304 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c95816-c63f-4e69-a329-197357242148 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:18.924694 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 986/1970] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:18.926647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b60ed6-d0ed-4bc9-a730-66e0e595a7de admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:18.939958 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b60ed6-d0ed-4bc9-a730-66e0e595a7de admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:18.946737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6452faf-49c0-4867-84da-11472846353f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:18.947450 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6452faf-49c0-4867-84da-11472846353f admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:18.947724 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6452faf-49c0-4867-84da-11472846353f admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:18.948032 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6452faf-49c0-4867-84da-11472846353f admin admin] PATH_INFO: `/v3/projects/a641327daaa04f398b094dab45352b30/users/036bcef8c5604c5b8e26420c3bb310ac/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:18.971271 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 985/1971] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:18 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:45:18.977630 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c95816-c63f-4e69-a329-197357242148 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:18.980973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6452faf-49c0-4867-84da-11472846353f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a641327daaa04f398b094dab45352b30, user_id=036bcef8c5604c5b8e26420c3bb310ac, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:18.985335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6452faf-49c0-4867-84da-11472846353f admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:19.022008 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 987/1972] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:18 2022] PUT /identity/v3/projects/a641327daaa04f398b094dab45352b30/users/036bcef8c5604c5b8e26420c3bb310ac/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:19.029993 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87d22e-420e-4e0f-8925-e1b3051f7dc5 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:19.030269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87d22e-420e-4e0f-8925-e1b3051f7dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:19.030550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87d22e-420e-4e0f-8925-e1b3051f7dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:19.077770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-8e87d22e-420e-4e0f-8925-e1b3051f7dc5 None None] MFA Rules not processed for user `036bcef8c5604c5b8e26420c3bb310ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:19.079735 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c95816-c63f-4e69-a329-197357242148 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:19.081421 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-8e87d22e-420e-4e0f-8925-e1b3051f7dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:19.124506 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:19.181529 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebec255-2305-4adc-a0c3-2499d5b7a212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:19.182597 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebec255-2305-4adc-a0c3-2499d5b7a212 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:19.183284 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebec255-2305-4adc-a0c3-2499d5b7a212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:19.183512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebec255-2305-4adc-a0c3-2499d5b7a212 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:19.185585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebec255-2305-4adc-a0c3-2499d5b7a212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:19.189464 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebec255-2305-4adc-a0c3-2499d5b7a212 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:19.201632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 988/1973] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:19.214065 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87d22e-420e-4e0f-8925-e1b3051f7dc5 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:19.215807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-8e87d22e-420e-4e0f-8925-e1b3051f7dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:19.233318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 986/1974] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:18 2022] POST /identity/v3/users => generated 295 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:45:19.235265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebec255-2305-4adc-a0c3-2499d5b7a212 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:19.254525 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebec255-2305-4adc-a0c3-2499d5b7a212 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:19.264927 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81844509-b9cb-40e9-8b2a-bc14b23984e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:19.266238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81844509-b9cb-40e9-8b2a-bc14b23984e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:19.266723 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81844509-b9cb-40e9-8b2a-bc14b23984e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:19.267226 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81844509-b9cb-40e9-8b2a-bc14b23984e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:19.268221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81844509-b9cb-40e9-8b2a-bc14b23984e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:19.269975 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81844509-b9cb-40e9-8b2a-bc14b23984e1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:19.283617 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 987/1975] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:45:19.289192 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81844509-b9cb-40e9-8b2a-bc14b23984e1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:19.297884 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87d22e-420e-4e0f-8925-e1b3051f7dc5 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:19.301454 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81844509-b9cb-40e9-8b2a-bc14b23984e1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:19.302557 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea3114-2a67-4edf-a2e0-cacf4a6706d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:19.303202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea3114-2a67-4edf-a2e0-cacf4a6706d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:19.303470 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea3114-2a67-4edf-a2e0-cacf4a6706d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:19.303749 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea3114-2a67-4edf-a2e0-cacf4a6706d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:19.306694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501699e-490b-4a6b-83f1-a8c5d547357f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:19.307401 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501699e-490b-4a6b-83f1-a8c5d547357f admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:19.307824 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501699e-490b-4a6b-83f1-a8c5d547357f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:19.308132 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501699e-490b-4a6b-83f1-a8c5d547357f admin admin] PATH_INFO: `/v3/projects/78ec4b688b1147f3b7fd39dad9357ab0/users/f946f8b5a3144be4bcae54c0aa79a6b2/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:19.315049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea3114-2a67-4edf-a2e0-cacf4a6706d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:19.317486 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea3114-2a67-4edf-a2e0-cacf4a6706d4 service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:19.328917 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 989/1976] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:45:19.332159 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501699e-490b-4a6b-83f1-a8c5d547357f admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ec4b688b1147f3b7fd39dad9357ab0, user_id=f946f8b5a3144be4bcae54c0aa79a6b2, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:19.336931 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501699e-490b-4a6b-83f1-a8c5d547357f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:19.360069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 988/1977] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:19 2022] PUT /identity/v3/projects/78ec4b688b1147f3b7fd39dad9357ab0/users/f946f8b5a3144be4bcae54c0aa79a6b2/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:19.372256 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea3114-2a67-4edf-a2e0-cacf4a6706d4 service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:19.385463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea3114-2a67-4edf-a2e0-cacf4a6706d4 service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:19.395862 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:19.465207 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08285b45-55c8-43d1-86dc-762d974ccee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:19.465843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08285b45-55c8-43d1-86dc-762d974ccee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:19.466179 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08285b45-55c8-43d1-86dc-762d974ccee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:19.466456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08285b45-55c8-43d1-86dc-762d974ccee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:19.467192 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08285b45-55c8-43d1-86dc-762d974ccee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:19.468938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08285b45-55c8-43d1-86dc-762d974ccee4 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:19.488761 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 990/1978] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:19.489077 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501699e-490b-4a6b-83f1-a8c5d547357f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:19.502758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501699e-490b-4a6b-83f1-a8c5d547357f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:19.521374 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0377c9-dd88-4e4a-be29-48634117396f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:19.521974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0377c9-dd88-4e4a-be29-48634117396f admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:19.522253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0377c9-dd88-4e4a-be29-48634117396f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:19.522519 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0377c9-dd88-4e4a-be29-48634117396f admin admin] PATH_INFO: `/v3/projects/78ec4b688b1147f3b7fd39dad9357ab0/users/f946f8b5a3144be4bcae54c0aa79a6b2/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:19.524755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0377c9-dd88-4e4a-be29-48634117396f admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ec4b688b1147f3b7fd39dad9357ab0, user_id=f946f8b5a3144be4bcae54c0aa79a6b2, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:19.528360 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0377c9-dd88-4e4a-be29-48634117396f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:19.551591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 989/1979] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:19 2022] PUT /identity/v3/projects/78ec4b688b1147f3b7fd39dad9357ab0/users/f946f8b5a3144be4bcae54c0aa79a6b2/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:45:19.564080 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927daea0-f2ad-46ef-b69d-e21f97401f55 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:19.564378 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927daea0-f2ad-46ef-b69d-e21f97401f55 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:19.564704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927daea0-f2ad-46ef-b69d-e21f97401f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:19.604142 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-927daea0-f2ad-46ef-b69d-e21f97401f55 None None] MFA Rules not processed for user `f946f8b5a3144be4bcae54c0aa79a6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:19.605699 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-927daea0-f2ad-46ef-b69d-e21f97401f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:19.727045 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 991/1980] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:19.736086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0377c9-dd88-4e4a-be29-48634117396f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:19.739585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1a0377c9-dd88-4e4a-be29-48634117396f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:19.843615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0377c9-dd88-4e4a-be29-48634117396f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:19.869034 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf694f-d4f4-406a-8fdc-9fe56d820d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:19.870128 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf694f-d4f4-406a-8fdc-9fe56d820d63 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:19.870568 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf694f-d4f4-406a-8fdc-9fe56d820d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:19.871097 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf694f-d4f4-406a-8fdc-9fe56d820d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:19.880201 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-afcf694f-d4f4-406a-8fdc-9fe56d820d63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:19.995353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf694f-d4f4-406a-8fdc-9fe56d820d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:19.996933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf694f-d4f4-406a-8fdc-9fe56d820d63 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:20.015123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 990/1981] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:20.130301 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927daea0-f2ad-46ef-b69d-e21f97401f55 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:20.134859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-927daea0-f2ad-46ef-b69d-e21f97401f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:20.315610 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927daea0-f2ad-46ef-b69d-e21f97401f55 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:20.327224 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62bddb-f146-4d0f-9592-9fc987d9162e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:20.327913 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62bddb-f146-4d0f-9592-9fc987d9162e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:20.328328 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62bddb-f146-4d0f-9592-9fc987d9162e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:20.328608 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62bddb-f146-4d0f-9592-9fc987d9162e admin admin] PATH_INFO: `/v3/users/81ba430b51dc4f379d957ae0ff0d117d` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:20.330018 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62bddb-f146-4d0f-9592-9fc987d9162e admin admin] RBAC: Authorizing `identity:delete_user(user_id=81ba430b51dc4f379d957ae0ff0d117d)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:20.332628 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62bddb-f146-4d0f-9592-9fc987d9162e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:20.438099 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:20.464170 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6e62bddb-f146-4d0f-9592-9fc987d9162e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81ba430b51dc4f379d957ae0ff0d117d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:20.465994 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6e62bddb-f146-4d0f-9592-9fc987d9162e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81ba430b51dc4f379d957ae0ff0d117d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:20.468427 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6e62bddb-f146-4d0f-9592-9fc987d9162e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81ba430b51dc4f379d957ae0ff0d117d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:20.501294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6e62bddb-f146-4d0f-9592-9fc987d9162e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81ba430b51dc4f379d957ae0ff0d117d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:20.535270 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6e62bddb-f146-4d0f-9592-9fc987d9162e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81ba430b51dc4f379d957ae0ff0d117d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:20.576060 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 992/1982] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:20 2022] DELETE /identity/v3/users/81ba430b51dc4f379d957ae0ff0d117d => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:20.587714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81d49d-6414-4b41-9c4d-8aaa18b182f0 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:20.588642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81d49d-6414-4b41-9c4d-8aaa18b182f0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:20.589430 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81d49d-6414-4b41-9c4d-8aaa18b182f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:20.683866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-3d81d49d-6414-4b41-9c4d-8aaa18b182f0 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:20.692336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3d81d49d-6414-4b41-9c4d-8aaa18b182f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:20.744578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:20.819798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 991/1983] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:20.834698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62bddb-f146-4d0f-9592-9fc987d9162e admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:20.836992 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6e62bddb-f146-4d0f-9592-9fc987d9162e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:20.925600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62bddb-f146-4d0f-9592-9fc987d9162e admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:20.930824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684ab3c-c4d5-4766-80e8-c53384bdf2d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:20.931555 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684ab3c-c4d5-4766-80e8-c53384bdf2d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:20.931923 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684ab3c-c4d5-4766-80e8-c53384bdf2d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:20.932229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684ab3c-c4d5-4766-80e8-c53384bdf2d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:20.945718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684ab3c-c4d5-4766-80e8-c53384bdf2d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:20.947179 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684ab3c-c4d5-4766-80e8-c53384bdf2d9 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:20.958247 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 993/1984] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:21.402562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81d49d-6414-4b41-9c4d-8aaa18b182f0 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:21.405142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3d81d49d-6414-4b41-9c4d-8aaa18b182f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:21.425602 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81d49d-6414-4b41-9c4d-8aaa18b182f0 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:21.433855 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9aaa10-a5b4-4965-8193-de78a5a4be0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:21.434597 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9aaa10-a5b4-4965-8193-de78a5a4be0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:21.437151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9aaa10-a5b4-4965-8193-de78a5a4be0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:21.437840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9aaa10-a5b4-4965-8193-de78a5a4be0b admin admin] PATH_INFO: `/v3/projects/a98c4c86a6c54cacb4b22668ce01ca93` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:21.439525 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9aaa10-a5b4-4965-8193-de78a5a4be0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a98c4c86a6c54cacb4b22668ce01ca93)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:21.441503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9aaa10-a5b4-4965-8193-de78a5a4be0b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:21.516776 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8b9aaa10-a5b4-4965-8193-de78a5a4be0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a98c4c86a6c54cacb4b22668ce01ca93', 'request_id': 'req-8b9aaa10-a5b4-4965-8193-de78a5a4be0b'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:21.526755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8b9aaa10-a5b4-4965-8193-de78a5a4be0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a98c4c86a6c54cacb4b22668ce01ca93', 'request_id': 'req-8b9aaa10-a5b4-4965-8193-de78a5a4be0b'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:21.538593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8b9aaa10-a5b4-4965-8193-de78a5a4be0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a98c4c86a6c54cacb4b22668ce01ca93', 'request_id': 'req-8b9aaa10-a5b4-4965-8193-de78a5a4be0b'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:21.554728 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8b9aaa10-a5b4-4965-8193-de78a5a4be0b admin admin] The token cache is being invalidate because project a98c4c86a6c54cacb4b22668ce01ca93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:21.555359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8b9aaa10-a5b4-4965-8193-de78a5a4be0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:21.556536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8b9aaa10-a5b4-4965-8193-de78a5a4be0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:21.559379 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 992/1985] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:21 2022] DELETE /identity/v3/projects/a98c4c86a6c54cacb4b22668ce01ca93 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:21.569555 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd3807-0555-42a5-8c31-1ec0d79d3d0b None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:21.569555 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd3807-0555-42a5-8c31-1ec0d79d3d0b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:21.570225 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd3807-0555-42a5-8c31-1ec0d79d3d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:21.597166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-8bbd3807-0555-42a5-8c31-1ec0d79d3d0b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:21.598217 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-8bbd3807-0555-42a5-8c31-1ec0d79d3d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:21.683871 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 994/1986] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:21.693194 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8df4a3-0017-448b-9ef0-650ef09f8aff None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:21.693661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8df4a3-0017-448b-9ef0-650ef09f8aff None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:21.694133 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8df4a3-0017-448b-9ef0-650ef09f8aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:21.716332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-bd8df4a3-0017-448b-9ef0-650ef09f8aff None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:21.717678 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bd8df4a3-0017-448b-9ef0-650ef09f8aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:21.736392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 993/1987] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:21.741757 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd3807-0555-42a5-8c31-1ec0d79d3d0b None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:21.753919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd3807-0555-42a5-8c31-1ec0d79d3d0b None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:21.765039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aad134-c007-43b7-bb9b-9a69b992611c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:21.765039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aad134-c007-43b7-bb9b-9a69b992611c admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:21.765039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aad134-c007-43b7-bb9b-9a69b992611c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:21.774038 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aad134-c007-43b7-bb9b-9a69b992611c admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:21.782686 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aad134-c007-43b7-bb9b-9a69b992611c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:45:21.783982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aad134-c007-43b7-bb9b-9a69b992611c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:21.790319 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aad134-c007-43b7-bb9b-9a69b992611c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:21.831175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 995/1988] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:45:21.841436 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8df4a3-0017-448b-9ef0-650ef09f8aff None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:21.886326 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8df4a3-0017-448b-9ef0-650ef09f8aff None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:21.908405 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda319c6-f826-4114-9da2-0545ec669bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:21.909385 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda319c6-f826-4114-9da2-0545ec669bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:21.909802 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda319c6-f826-4114-9da2-0545ec669bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:21.910324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda319c6-f826-4114-9da2-0545ec669bd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:21.915606 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda319c6-f826-4114-9da2-0545ec669bd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:21.924152 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda319c6-f826-4114-9da2-0545ec669bd1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:22.047219 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aad134-c007-43b7-bb9b-9a69b992611c admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:22.050423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c4aad134-c007-43b7-bb9b-9a69b992611c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:22.052460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 994/1989] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:21 2022] POST /identity/v3/projects => generated 363 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:22.053953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda319c6-f826-4114-9da2-0545ec669bd1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:22.173291 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:22.192866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda319c6-f826-4114-9da2-0545ec669bd1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:22.200534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b9abc-789d-4784-baff-d07fee0f228e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:22.201593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b9abc-789d-4784-baff-d07fee0f228e admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:22.202099 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b9abc-789d-4784-baff-d07fee0f228e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:22.202633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b9abc-789d-4784-baff-d07fee0f228e admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:22.202815 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aad134-c007-43b7-bb9b-9a69b992611c admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:22.203697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b9abc-789d-4784-baff-d07fee0f228e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:22.205994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b9abc-789d-4784-baff-d07fee0f228e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:22.241676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 995/1990] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:22 2022] POST /identity/v3/users => generated 282 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:22.243195 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b9abc-789d-4784-baff-d07fee0f228e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:22.247139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bf5b9abc-789d-4784-baff-d07fee0f228e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:22.274360 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b9abc-789d-4784-baff-d07fee0f228e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:22.292916 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c3fd7-e762-4797-95d6-74e284a90691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:22.296638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c3fd7-e762-4797-95d6-74e284a90691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:22.297368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c3fd7-e762-4797-95d6-74e284a90691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:22.300651 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c3fd7-e762-4797-95d6-74e284a90691 admin admin] PATH_INFO: `/v3/users/c657f91b5fbf4d9b9c79c560c8c3e3e4` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:22.306422 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c3fd7-e762-4797-95d6-74e284a90691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c657f91b5fbf4d9b9c79c560c8c3e3e4)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:22.313240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c3fd7-e762-4797-95d6-74e284a90691 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:22.330343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:22.343407 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5726071-764a-4b7d-b221-97ae3aa1cb34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:22.345900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5726071-764a-4b7d-b221-97ae3aa1cb34 service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:22.346708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5726071-764a-4b7d-b221-97ae3aa1cb34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:22.348445 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5726071-764a-4b7d-b221-97ae3aa1cb34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:22.352536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c5726071-764a-4b7d-b221-97ae3aa1cb34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:22.399246 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e62c3fd7-e762-4797-95d6-74e284a90691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c657f91b5fbf4d9b9c79c560c8c3e3e4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:22.411818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e62c3fd7-e762-4797-95d6-74e284a90691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c657f91b5fbf4d9b9c79c560c8c3e3e4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:22.425829 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e62c3fd7-e762-4797-95d6-74e284a90691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c657f91b5fbf4d9b9c79c560c8c3e3e4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:22.427660 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e62c3fd7-e762-4797-95d6-74e284a90691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c657f91b5fbf4d9b9c79c560c8c3e3e4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:22.457467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e62c3fd7-e762-4797-95d6-74e284a90691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c657f91b5fbf4d9b9c79c560c8c3e3e4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:22.461368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5726071-764a-4b7d-b221-97ae3aa1cb34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:22.464819 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5726071-764a-4b7d-b221-97ae3aa1cb34 service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:22.480221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 996/1991] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:22 2022] DELETE /identity/v3/users/c657f91b5fbf4d9b9c79c560c8c3e3e4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:22.480221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c3fd7-e762-4797-95d6-74e284a90691 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:22.480221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e62c3fd7-e762-4797-95d6-74e284a90691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:22.493517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 996/1992] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:22.496556 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f962293-017a-4766-917a-b565abe0321a None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:22.497154 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f962293-017a-4766-917a-b565abe0321a None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:22.497587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f962293-017a-4766-917a-b565abe0321a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:22.533235 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-6f962293-017a-4766-917a-b565abe0321a None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:22.535856 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6f962293-017a-4766-917a-b565abe0321a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:22.603433 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c3fd7-e762-4797-95d6-74e284a90691 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:22.618119 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66320c5-5cac-404d-a3e1-56ff39400929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:22.619178 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66320c5-5cac-404d-a3e1-56ff39400929 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:22.619660 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66320c5-5cac-404d-a3e1-56ff39400929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:22.620194 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66320c5-5cac-404d-a3e1-56ff39400929 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:22.626949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66320c5-5cac-404d-a3e1-56ff39400929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:22.629262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66320c5-5cac-404d-a3e1-56ff39400929 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:22.667182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 997/1993] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:22.674100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66320c5-5cac-404d-a3e1-56ff39400929 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:22.702104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66320c5-5cac-404d-a3e1-56ff39400929 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:22.711366 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 997/1994] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:22.713594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13b184-927b-4aec-ab46-4a746a2a25aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:22.714833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13b184-927b-4aec-ab46-4a746a2a25aa admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:22.715463 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13b184-927b-4aec-ab46-4a746a2a25aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:22.716260 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13b184-927b-4aec-ab46-4a746a2a25aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:22.717696 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a13b184-927b-4aec-ab46-4a746a2a25aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:22.721600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f962293-017a-4766-917a-b565abe0321a None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:22.721807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a13b184-927b-4aec-ab46-4a746a2a25aa admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:22.725220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6f962293-017a-4766-917a-b565abe0321a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:22.754160 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 998/1995] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:45:22.756403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13b184-927b-4aec-ab46-4a746a2a25aa admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:22.774759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13b184-927b-4aec-ab46-4a746a2a25aa admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:22.784708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb94422-20af-4c61-ba50-8176fa5a30ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:22.786337 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb94422-20af-4c61-ba50-8176fa5a30ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:22.787355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb94422-20af-4c61-ba50-8176fa5a30ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:22.787640 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb94422-20af-4c61-ba50-8176fa5a30ce admin admin] PATH_INFO: `/v3/projects/fa21c1ec3db54b12b2965e81be297713/users/6b76f320445b4723908a44915f879add/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:22.814093 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f962293-017a-4766-917a-b565abe0321a None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:22.817363 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb94422-20af-4c61-ba50-8176fa5a30ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa21c1ec3db54b12b2965e81be297713, user_id=6b76f320445b4723908a44915f879add, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:22.819093 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7f812-24be-47ad-a588-015b0a618e38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:22.820027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7f812-24be-47ad-a588-015b0a618e38 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:22.820629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7f812-24be-47ad-a588-015b0a618e38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:22.821226 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7f812-24be-47ad-a588-015b0a618e38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:22.823258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb94422-20af-4c61-ba50-8176fa5a30ce admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:22.846625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:22.867820 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 999/1996] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:22 2022] PUT /identity/v3/projects/fa21c1ec3db54b12b2965e81be297713/users/6b76f320445b4723908a44915f879add/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:22.868076 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7f812-24be-47ad-a588-015b0a618e38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:22.868076 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7f812-24be-47ad-a588-015b0a618e38 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:22.871650 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d2b4e-988c-4c19-9de7-c1e5b06add0d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:22.872551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d2b4e-988c-4c19-9de7-c1e5b06add0d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:22.873162 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d2b4e-988c-4c19-9de7-c1e5b06add0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:22.909586 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 998/1997] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:22.932480 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-385d2b4e-988c-4c19-9de7-c1e5b06add0d None None] MFA Rules not processed for user `6b76f320445b4723908a44915f879add`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:22.934720 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-385d2b4e-988c-4c19-9de7-c1e5b06add0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:22.986092 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:23.056748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1000/1998] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:23.057588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7f812-24be-47ad-a588-015b0a618e38 service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:23.059131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0ae7f812-24be-47ad-a588-015b0a618e38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:23.172634 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7f812-24be-47ad-a588-015b0a618e38 service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:23.244393 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d04124-9035-4121-b209-fe7496207c7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:23.245081 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d04124-9035-4121-b209-fe7496207c7c service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:23.245423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d04124-9035-4121-b209-fe7496207c7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:23.245690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d04124-9035-4121-b209-fe7496207c7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:23.257253 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d04124-9035-4121-b209-fe7496207c7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:23.259108 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d04124-9035-4121-b209-fe7496207c7c service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:23.273039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 999/1999] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:45:23.664701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d2b4e-988c-4c19-9de7-c1e5b06add0d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:23.668098 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-385d2b4e-988c-4c19-9de7-c1e5b06add0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:23.823217 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d2b4e-988c-4c19-9de7-c1e5b06add0d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:23.829677 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf7466-1792-4af2-b4d1-55baf7966b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:23.830423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf7466-1792-4af2-b4d1-55baf7966b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:23.830722 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf7466-1792-4af2-b4d1-55baf7966b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:23.831024 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf7466-1792-4af2-b4d1-55baf7966b9c admin admin] PATH_INFO: `/v3/projects/952be476569e4bf9abd7cf2cb315cb9a` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:23.832333 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf7466-1792-4af2-b4d1-55baf7966b9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=952be476569e4bf9abd7cf2cb315cb9a)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:23.834720 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf7466-1792-4af2-b4d1-55baf7966b9c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:23.919258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0aaf7466-1792-4af2-b4d1-55baf7966b9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '952be476569e4bf9abd7cf2cb315cb9a', 'request_id': 'req-0aaf7466-1792-4af2-b4d1-55baf7966b9c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:23.932257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0aaf7466-1792-4af2-b4d1-55baf7966b9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '952be476569e4bf9abd7cf2cb315cb9a', 'request_id': 'req-0aaf7466-1792-4af2-b4d1-55baf7966b9c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:23.946690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0aaf7466-1792-4af2-b4d1-55baf7966b9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '952be476569e4bf9abd7cf2cb315cb9a', 'request_id': 'req-0aaf7466-1792-4af2-b4d1-55baf7966b9c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:23.962025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0aaf7466-1792-4af2-b4d1-55baf7966b9c admin admin] The token cache is being invalidate because project 952be476569e4bf9abd7cf2cb315cb9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:23.962451 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0aaf7466-1792-4af2-b4d1-55baf7966b9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:23.963454 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0aaf7466-1792-4af2-b4d1-55baf7966b9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:23.965612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1001/2000] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:23 2022] DELETE /identity/v3/projects/952be476569e4bf9abd7cf2cb315cb9a => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:45:23.970555 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d04124-9035-4121-b209-fe7496207c7c service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:23.982081 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d04124-9035-4121-b209-fe7496207c7c service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:24.085626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f2f69-e820-4ce8-ae8e-491adfd03b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:24.086781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f2f69-e820-4ce8-ae8e-491adfd03b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:24.087291 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f2f69-e820-4ce8-ae8e-491adfd03b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:24.087796 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f2f69-e820-4ce8-ae8e-491adfd03b5e admin admin] PATH_INFO: `/v3/users/25ed9f03e11140a280a2f107fc9aafa2` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:24.089991 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f2f69-e820-4ce8-ae8e-491adfd03b5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=25ed9f03e11140a280a2f107fc9aafa2)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:24.094058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f2f69-e820-4ce8-ae8e-491adfd03b5e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:24.185033 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b15f2f69-e820-4ce8-ae8e-491adfd03b5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25ed9f03e11140a280a2f107fc9aafa2'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:24.187321 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b15f2f69-e820-4ce8-ae8e-491adfd03b5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25ed9f03e11140a280a2f107fc9aafa2'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:24.188276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b15f2f69-e820-4ce8-ae8e-491adfd03b5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25ed9f03e11140a280a2f107fc9aafa2'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:24.199652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b15f2f69-e820-4ce8-ae8e-491adfd03b5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25ed9f03e11140a280a2f107fc9aafa2'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:24.203188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:24.220149 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b15f2f69-e820-4ce8-ae8e-491adfd03b5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25ed9f03e11140a280a2f107fc9aafa2'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:24.235557 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1000/2001] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:23 2022] DELETE /identity/v3/users/25ed9f03e11140a280a2f107fc9aafa2 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:45:24.890134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf7466-1792-4af2-b4d1-55baf7966b9c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:24.892229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0aaf7466-1792-4af2-b4d1-55baf7966b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:24.918340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf7466-1792-4af2-b4d1-55baf7966b9c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:25.049374 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294f36e-94e4-417f-b254-a01e42976599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:25.054327 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294f36e-94e4-417f-b254-a01e42976599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:25.054900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294f36e-94e4-417f-b254-a01e42976599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:25.056623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294f36e-94e4-417f-b254-a01e42976599 admin admin] PATH_INFO: `/v3/projects/b45627ad48a14d4d9b2ca63774042e13` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:25.059230 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f294f36e-94e4-417f-b254-a01e42976599 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b45627ad48a14d4d9b2ca63774042e13)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:25.064979 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f294f36e-94e4-417f-b254-a01e42976599 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:25.079986 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:25.152800 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f294f36e-94e4-417f-b254-a01e42976599 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b45627ad48a14d4d9b2ca63774042e13', 'request_id': 'req-f294f36e-94e4-417f-b254-a01e42976599'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:25.166530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f294f36e-94e4-417f-b254-a01e42976599 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b45627ad48a14d4d9b2ca63774042e13', 'request_id': 'req-f294f36e-94e4-417f-b254-a01e42976599'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:25.176280 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f294f36e-94e4-417f-b254-a01e42976599 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b45627ad48a14d4d9b2ca63774042e13', 'request_id': 'req-f294f36e-94e4-417f-b254-a01e42976599'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:25.203103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f294f36e-94e4-417f-b254-a01e42976599 admin admin] The token cache is being invalidate because project b45627ad48a14d4d9b2ca63774042e13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:25.204122 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f294f36e-94e4-417f-b254-a01e42976599 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:25.205828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f294f36e-94e4-417f-b254-a01e42976599 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:25.211714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1002/2002] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:24 2022] DELETE /identity/v3/projects/b45627ad48a14d4d9b2ca63774042e13 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:25.213765 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdc256-8e6c-4466-8223-497726248133 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:25.221421 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdc256-8e6c-4466-8223-497726248133 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:25.222949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdc256-8e6c-4466-8223-497726248133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:25.231358 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294f36e-94e4-417f-b254-a01e42976599 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:25.234204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f294f36e-94e4-417f-b254-a01e42976599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:25.255809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-acfdc256-8e6c-4466-8223-497726248133 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:25.257968 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-acfdc256-8e6c-4466-8223-497726248133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:25.353114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1001/2003] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:25.354683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6874c1-3d6a-4731-9657-608b3926d893 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:25.354974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6874c1-3d6a-4731-9657-608b3926d893 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:25.355252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6874c1-3d6a-4731-9657-608b3926d893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:25.357775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294f36e-94e4-417f-b254-a01e42976599 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:25.363531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a351b0a-b3f8-4229-a48a-d97c0d27b64e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:25.364598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a351b0a-b3f8-4229-a48a-d97c0d27b64e service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:25.364975 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a351b0a-b3f8-4229-a48a-d97c0d27b64e service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:25.365342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a351b0a-b3f8-4229-a48a-d97c0d27b64e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:25.367968 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5a351b0a-b3f8-4229-a48a-d97c0d27b64e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:25.381165 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-8c6874c1-3d6a-4731-9657-608b3926d893 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:25.383327 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-8c6874c1-3d6a-4731-9657-608b3926d893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:25.399818 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1002/2004] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:25.401432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6874c1-3d6a-4731-9657-608b3926d893 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:25.416134 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6874c1-3d6a-4731-9657-608b3926d893 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:25.421371 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090e157-8889-46e1-9c04-2311913f8777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:25.422045 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090e157-8889-46e1-9c04-2311913f8777 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:25.422402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090e157-8889-46e1-9c04-2311913f8777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:25.422563 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090e157-8889-46e1-9c04-2311913f8777 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:25.423229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090e157-8889-46e1-9c04-2311913f8777 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:45:25.423530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090e157-8889-46e1-9c04-2311913f8777 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:25.425243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090e157-8889-46e1-9c04-2311913f8777 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:25.445392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1003/2005] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:45:25.451200 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090e157-8889-46e1-9c04-2311913f8777 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:25.469020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090e157-8889-46e1-9c04-2311913f8777 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:25.476821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013652b8-6c17-4cb1-80fa-9f1e63ae63cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:25.477795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013652b8-6c17-4cb1-80fa-9f1e63ae63cc admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:25.478279 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013652b8-6c17-4cb1-80fa-9f1e63ae63cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:25.479204 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013652b8-6c17-4cb1-80fa-9f1e63ae63cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:25.479641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013652b8-6c17-4cb1-80fa-9f1e63ae63cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:25.483142 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013652b8-6c17-4cb1-80fa-9f1e63ae63cc admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:25.483473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a351b0a-b3f8-4229-a48a-d97c0d27b64e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:25.487119 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a351b0a-b3f8-4229-a48a-d97c0d27b64e service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:25.504868 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1003/2006] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:25.525269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1004/2007] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:25 2022] POST /identity/v3/projects => generated 387 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:25.531025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a351b0a-b3f8-4229-a48a-d97c0d27b64e service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:25.543263 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a351b0a-b3f8-4229-a48a-d97c0d27b64e service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:25.646901 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:25.674900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999a43f-00ed-4648-9ea1-b3c2cd73af26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:25.675694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999a43f-00ed-4648-9ea1-b3c2cd73af26 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:25.676118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999a43f-00ed-4648-9ea1-b3c2cd73af26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:25.676493 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999a43f-00ed-4648-9ea1-b3c2cd73af26 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:25.677241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999a43f-00ed-4648-9ea1-b3c2cd73af26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:25.679293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999a43f-00ed-4648-9ea1-b3c2cd73af26 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:25.707263 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1004/2008] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:25 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:25.711467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013652b8-6c17-4cb1-80fa-9f1e63ae63cc admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:25.724575 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013652b8-6c17-4cb1-80fa-9f1e63ae63cc admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:25.729624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f70662-a71c-42bb-afd2-e8f6c6cea8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:25.730264 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f70662-a71c-42bb-afd2-e8f6c6cea8f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:25.730565 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f70662-a71c-42bb-afd2-e8f6c6cea8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:25.730866 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f70662-a71c-42bb-afd2-e8f6c6cea8f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:25.731563 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f70662-a71c-42bb-afd2-e8f6c6cea8f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:25.733374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f70662-a71c-42bb-afd2-e8f6c6cea8f2 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:25.746839 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1005/2009] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:45:25.750933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999a43f-00ed-4648-9ea1-b3c2cd73af26 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:25.763482 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999a43f-00ed-4648-9ea1-b3c2cd73af26 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:25.769058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc5e92-1ce9-4ce5-9f42-32059063ace5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:25.769705 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc5e92-1ce9-4ce5-9f42-32059063ace5 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:25.770002 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc5e92-1ce9-4ce5-9f42-32059063ace5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:25.770254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc5e92-1ce9-4ce5-9f42-32059063ace5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:25.770929 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc5e92-1ce9-4ce5-9f42-32059063ace5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:25.772631 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc5e92-1ce9-4ce5-9f42-32059063ace5 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:25.785608 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1005/2010] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:25.789827 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f70662-a71c-42bb-afd2-e8f6c6cea8f2 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:25.802362 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f70662-a71c-42bb-afd2-e8f6c6cea8f2 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:25.807970 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79759259-ab6b-4c19-a5ca-10bda8e3c72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:25.808928 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79759259-ab6b-4c19-a5ca-10bda8e3c72c admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:25.809357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79759259-ab6b-4c19-a5ca-10bda8e3c72c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:25.809863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79759259-ab6b-4c19-a5ca-10bda8e3c72c admin admin] PATH_INFO: `/v3/projects/92e0daceadad4dba9d826561fee9bbec/users/807e6475892348f4a13995ce895e0faf/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:25.837047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79759259-ab6b-4c19-a5ca-10bda8e3c72c admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e0daceadad4dba9d826561fee9bbec, user_id=807e6475892348f4a13995ce895e0faf, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:25.839985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79759259-ab6b-4c19-a5ca-10bda8e3c72c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:25.859146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1006/2011] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:25 2022] PUT /identity/v3/projects/92e0daceadad4dba9d826561fee9bbec/users/807e6475892348f4a13995ce895e0faf/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:25.870175 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e27a1f-d31b-4ef4-98f0-ef9cf9429c0a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:25.870436 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e27a1f-d31b-4ef4-98f0-ef9cf9429c0a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:25.870720 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e27a1f-d31b-4ef4-98f0-ef9cf9429c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:25.917091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-98e27a1f-d31b-4ef4-98f0-ef9cf9429c0a None None] MFA Rules not processed for user `807e6475892348f4a13995ce895e0faf`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:25.918626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-98e27a1f-d31b-4ef4-98f0-ef9cf9429c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:26.030778 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1006/2012] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:26.036396 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79759259-ab6b-4c19-a5ca-10bda8e3c72c admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:26.136853 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:26.143874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79759259-ab6b-4c19-a5ca-10bda8e3c72c admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:26.152963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b81de6-3a4b-4390-8613-641dcca66e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:26.153877 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b81de6-3a4b-4390-8613-641dcca66e74 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:26.154533 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b81de6-3a4b-4390-8613-641dcca66e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:26.155284 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b81de6-3a4b-4390-8613-641dcca66e74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:26.156208 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b81de6-3a4b-4390-8613-641dcca66e74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:26.158491 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b81de6-3a4b-4390-8613-641dcca66e74 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:26.206356 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1007/2013] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:26 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:45:26.206856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e27a1f-d31b-4ef4-98f0-ef9cf9429c0a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:26.222613 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e27a1f-d31b-4ef4-98f0-ef9cf9429c0a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:26.367960 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cc99e-f6a7-4dbb-bda4-2d56a607c83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:26.369313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cc99e-f6a7-4dbb-bda4-2d56a607c83e admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:26.369803 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cc99e-f6a7-4dbb-bda4-2d56a607c83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:26.370340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cc99e-f6a7-4dbb-bda4-2d56a607c83e admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:26.371832 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cc99e-f6a7-4dbb-bda4-2d56a607c83e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:26.376485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cc99e-f6a7-4dbb-bda4-2d56a607c83e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:26.428044 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1007/2014] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:26 2022] POST /identity/v3/users => generated 300 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:26.434266 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b81de6-3a4b-4390-8613-641dcca66e74 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:26.458723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b81de6-3a4b-4390-8613-641dcca66e74 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:26.467983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc26b1-b41b-478f-ae8a-23f0fc22c5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:26.469164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc26b1-b41b-478f-ae8a-23f0fc22c5fd admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:26.469626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc26b1-b41b-478f-ae8a-23f0fc22c5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:26.470123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc26b1-b41b-478f-ae8a-23f0fc22c5fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:26.471360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc26b1-b41b-478f-ae8a-23f0fc22c5fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:26.474501 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc26b1-b41b-478f-ae8a-23f0fc22c5fd admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:26.498333 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1008/2015] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:26.505956 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cc99e-f6a7-4dbb-bda4-2d56a607c83e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:26.517382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cc99e-f6a7-4dbb-bda4-2d56a607c83e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:26.523132 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec60c5-1dbd-4535-ac84-3041efd9bcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:26.524126 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec60c5-1dbd-4535-ac84-3041efd9bcc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:26.524462 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec60c5-1dbd-4535-ac84-3041efd9bcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:26.524793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec60c5-1dbd-4535-ac84-3041efd9bcc9 admin admin] PATH_INFO: `/v3/projects/2a1fe0bd68a4453c9a1bbdcaaaadb190/users/663b75b4c17d4ae2ad0599fe7e0795ef/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:26.553178 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec60c5-1dbd-4535-ac84-3041efd9bcc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a1fe0bd68a4453c9a1bbdcaaaadb190, user_id=663b75b4c17d4ae2ad0599fe7e0795ef, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:26.555001 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec60c5-1dbd-4535-ac84-3041efd9bcc9 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:26.579920 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1008/2016] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:26 2022] PUT /identity/v3/projects/2a1fe0bd68a4453c9a1bbdcaaaadb190/users/663b75b4c17d4ae2ad0599fe7e0795ef/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:26.585954 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc26b1-b41b-478f-ae8a-23f0fc22c5fd admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:26.604645 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc26b1-b41b-478f-ae8a-23f0fc22c5fd admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:26.705533 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e07f43-d42c-42c1-8781-0158dde306a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:26.706324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e07f43-d42c-42c1-8781-0158dde306a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:26.706781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e07f43-d42c-42c1-8781-0158dde306a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:26.707098 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e07f43-d42c-42c1-8781-0158dde306a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:26.707982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e07f43-d42c-42c1-8781-0158dde306a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:26.710876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e07f43-d42c-42c1-8781-0158dde306a9 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:26.727595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1009/2017] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:45:26.732538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec60c5-1dbd-4535-ac84-3041efd9bcc9 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:26.743721 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec60c5-1dbd-4535-ac84-3041efd9bcc9 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:26.748902 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402b9b2-d9a8-46f4-8d5d-3749070f3223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:26.749599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402b9b2-d9a8-46f4-8d5d-3749070f3223 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:26.749879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402b9b2-d9a8-46f4-8d5d-3749070f3223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:26.750147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402b9b2-d9a8-46f4-8d5d-3749070f3223 admin admin] PATH_INFO: `/v3/projects/2a1fe0bd68a4453c9a1bbdcaaaadb190/users/663b75b4c17d4ae2ad0599fe7e0795ef/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:26.752421 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402b9b2-d9a8-46f4-8d5d-3749070f3223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a1fe0bd68a4453c9a1bbdcaaaadb190, user_id=663b75b4c17d4ae2ad0599fe7e0795ef, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:26.755126 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402b9b2-d9a8-46f4-8d5d-3749070f3223 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:26.767710 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1009/2018] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:26 2022] PUT /identity/v3/projects/2a1fe0bd68a4453c9a1bbdcaaaadb190/users/663b75b4c17d4ae2ad0599fe7e0795ef/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:45:26.779185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d9220-cd14-4a85-9e67-367a7d4fa7c9 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:26.779520 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d9220-cd14-4a85-9e67-367a7d4fa7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:26.779851 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d9220-cd14-4a85-9e67-367a7d4fa7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:26.818687 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-507d9220-cd14-4a85-9e67-367a7d4fa7c9 None None] MFA Rules not processed for user `663b75b4c17d4ae2ad0599fe7e0795ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:26.819934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-507d9220-cd14-4a85-9e67-367a7d4fa7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:26.909437 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1010/2019] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:26.926658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402b9b2-d9a8-46f4-8d5d-3749070f3223 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:26.928698 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8402b9b2-d9a8-46f4-8d5d-3749070f3223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:26.937225 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:27.036678 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402b9b2-d9a8-46f4-8d5d-3749070f3223 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:27.045777 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d2d3d-a4f1-4fd8-8210-574647cef4cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:27.047277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d2d3d-a4f1-4fd8-8210-574647cef4cf service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:27.048029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d2d3d-a4f1-4fd8-8210-574647cef4cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:27.048508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d2d3d-a4f1-4fd8-8210-574647cef4cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:27.066641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d2d3d-a4f1-4fd8-8210-574647cef4cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:27.069520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d2d3d-a4f1-4fd8-8210-574647cef4cf service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:27.084131 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1010/2020] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:27.382806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d9220-cd14-4a85-9e67-367a7d4fa7c9 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:27.470287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:27.497946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d9220-cd14-4a85-9e67-367a7d4fa7c9 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:27.511788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013d620-6fcf-4337-b3e1-c4e7de7a6d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:27.512580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013d620-6fcf-4337-b3e1-c4e7de7a6d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:27.512973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013d620-6fcf-4337-b3e1-c4e7de7a6d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:27.513305 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013d620-6fcf-4337-b3e1-c4e7de7a6d10 admin admin] PATH_INFO: `/v3/users/807e6475892348f4a13995ce895e0faf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:27.514758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013d620-6fcf-4337-b3e1-c4e7de7a6d10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=807e6475892348f4a13995ce895e0faf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:27.517117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013d620-6fcf-4337-b3e1-c4e7de7a6d10 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:27.598166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-3013d620-6fcf-4337-b3e1-c4e7de7a6d10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '807e6475892348f4a13995ce895e0faf'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:27.599352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-3013d620-6fcf-4337-b3e1-c4e7de7a6d10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '807e6475892348f4a13995ce895e0faf'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:27.599956 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-3013d620-6fcf-4337-b3e1-c4e7de7a6d10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '807e6475892348f4a13995ce895e0faf'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:27.611924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-3013d620-6fcf-4337-b3e1-c4e7de7a6d10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '807e6475892348f4a13995ce895e0faf'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:27.659466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-3013d620-6fcf-4337-b3e1-c4e7de7a6d10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '807e6475892348f4a13995ce895e0faf'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:27.693476 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1011/2021] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:27 2022] DELETE /identity/v3/users/807e6475892348f4a13995ce895e0faf => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:27.695303 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df525a-4a5d-4d6b-85cd-d44da2420c81 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:27.695634 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df525a-4a5d-4d6b-85cd-d44da2420c81 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:27.695938 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df525a-4a5d-4d6b-85cd-d44da2420c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:27.718555 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a4df525a-4a5d-4d6b-85cd-d44da2420c81 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:27.719849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a4df525a-4a5d-4d6b-85cd-d44da2420c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:27.815326 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1011/2022] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:27.827736 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013d620-6fcf-4337-b3e1-c4e7de7a6d10 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:27.829459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-3013d620-6fcf-4337-b3e1-c4e7de7a6d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:27.920311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013d620-6fcf-4337-b3e1-c4e7de7a6d10 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:27.925497 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3ede9-620d-4072-a64e-ede12b67799c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:27.926583 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3ede9-620d-4072-a64e-ede12b67799c service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:27.926852 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3ede9-620d-4072-a64e-ede12b67799c service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:27.927154 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3ede9-620d-4072-a64e-ede12b67799c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:27.939022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d3ede9-620d-4072-a64e-ede12b67799c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:27.940531 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d3ede9-620d-4072-a64e-ede12b67799c service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:27.951952 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1012/2023] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:28.471806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df525a-4a5d-4d6b-85cd-d44da2420c81 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:28.474913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a4df525a-4a5d-4d6b-85cd-d44da2420c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:28.499459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df525a-4a5d-4d6b-85cd-d44da2420c81 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:28.507168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ef58f-3759-4b6e-a9c1-8447fab39772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:28.508351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ef58f-3759-4b6e-a9c1-8447fab39772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:28.508787 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ef58f-3759-4b6e-a9c1-8447fab39772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:28.509240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ef58f-3759-4b6e-a9c1-8447fab39772 admin admin] PATH_INFO: `/v3/projects/92e0daceadad4dba9d826561fee9bbec` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:28.511336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ef58f-3759-4b6e-a9c1-8447fab39772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92e0daceadad4dba9d826561fee9bbec)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:28.514296 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ef58f-3759-4b6e-a9c1-8447fab39772 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:28.592708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2f0ef58f-3759-4b6e-a9c1-8447fab39772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92e0daceadad4dba9d826561fee9bbec', 'request_id': 'req-2f0ef58f-3759-4b6e-a9c1-8447fab39772'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:28.602530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2f0ef58f-3759-4b6e-a9c1-8447fab39772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92e0daceadad4dba9d826561fee9bbec', 'request_id': 'req-2f0ef58f-3759-4b6e-a9c1-8447fab39772'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:28.617486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2f0ef58f-3759-4b6e-a9c1-8447fab39772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92e0daceadad4dba9d826561fee9bbec', 'request_id': 'req-2f0ef58f-3759-4b6e-a9c1-8447fab39772'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:28.642504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2f0ef58f-3759-4b6e-a9c1-8447fab39772 admin admin] The token cache is being invalidate because project 92e0daceadad4dba9d826561fee9bbec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:28.643026 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2f0ef58f-3759-4b6e-a9c1-8447fab39772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:28.644371 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2f0ef58f-3759-4b6e-a9c1-8447fab39772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:28.647065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1012/2024] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:28 2022] DELETE /identity/v3/projects/92e0daceadad4dba9d826561fee9bbec => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:28.653227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3ede9-620d-4072-a64e-ede12b67799c service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:28.666764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3ede9-620d-4072-a64e-ede12b67799c service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:28.788175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece90769-a2d6-475c-b39a-975de515b78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:28.789298 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece90769-a2d6-475c-b39a-975de515b78b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:28.789773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece90769-a2d6-475c-b39a-975de515b78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:28.790264 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece90769-a2d6-475c-b39a-975de515b78b admin admin] PATH_INFO: `/v3/users/663b75b4c17d4ae2ad0599fe7e0795ef` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:28.792548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece90769-a2d6-475c-b39a-975de515b78b admin admin] RBAC: Authorizing `identity:delete_user(user_id=663b75b4c17d4ae2ad0599fe7e0795ef)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:28.797397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece90769-a2d6-475c-b39a-975de515b78b admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:28.832375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:28.906953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-ece90769-a2d6-475c-b39a-975de515b78b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '663b75b4c17d4ae2ad0599fe7e0795ef'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:28.910884 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-ece90769-a2d6-475c-b39a-975de515b78b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '663b75b4c17d4ae2ad0599fe7e0795ef'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:28.914170 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-ece90769-a2d6-475c-b39a-975de515b78b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '663b75b4c17d4ae2ad0599fe7e0795ef'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:28.935876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-ece90769-a2d6-475c-b39a-975de515b78b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '663b75b4c17d4ae2ad0599fe7e0795ef'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:28.987436 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-ece90769-a2d6-475c-b39a-975de515b78b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '663b75b4c17d4ae2ad0599fe7e0795ef'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:29.016586 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1013/2025] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:28 2022] DELETE /identity/v3/users/663b75b4c17d4ae2ad0599fe7e0795ef => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:29.431172 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ef58f-3759-4b6e-a9c1-8447fab39772 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:29.433286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2f0ef58f-3759-4b6e-a9c1-8447fab39772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:29.450568 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ef58f-3759-4b6e-a9c1-8447fab39772 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:29.487814 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:29.531304 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072647e-6d70-4b47-981b-9ba4939518ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:29.531992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072647e-6d70-4b47-981b-9ba4939518ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:29.532305 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072647e-6d70-4b47-981b-9ba4939518ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:29.532574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072647e-6d70-4b47-981b-9ba4939518ed admin admin] PATH_INFO: `/v3/projects/2a1fe0bd68a4453c9a1bbdcaaaadb190` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:29.533869 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072647e-6d70-4b47-981b-9ba4939518ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a1fe0bd68a4453c9a1bbdcaaaadb190)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:29.535856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072647e-6d70-4b47-981b-9ba4939518ed admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:29.597710 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b072647e-6d70-4b47-981b-9ba4939518ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a1fe0bd68a4453c9a1bbdcaaaadb190', 'request_id': 'req-b072647e-6d70-4b47-981b-9ba4939518ed'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:29.611243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b072647e-6d70-4b47-981b-9ba4939518ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a1fe0bd68a4453c9a1bbdcaaaadb190', 'request_id': 'req-b072647e-6d70-4b47-981b-9ba4939518ed'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:29.621795 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b072647e-6d70-4b47-981b-9ba4939518ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a1fe0bd68a4453c9a1bbdcaaaadb190', 'request_id': 'req-b072647e-6d70-4b47-981b-9ba4939518ed'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:29.642587 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b072647e-6d70-4b47-981b-9ba4939518ed admin admin] The token cache is being invalidate because project 2a1fe0bd68a4453c9a1bbdcaaaadb190 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:29.642984 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b072647e-6d70-4b47-981b-9ba4939518ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:29.644098 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b072647e-6d70-4b47-981b-9ba4939518ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:29.646709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1013/2026] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:29 2022] DELETE /identity/v3/projects/2a1fe0bd68a4453c9a1bbdcaaaadb190 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:29.655451 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a0db7-0ad9-4556-b402-dedda3af71d1 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:29.656066 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a0db7-0ad9-4556-b402-dedda3af71d1 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:29.656066 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a0db7-0ad9-4556-b402-dedda3af71d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:29.689860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a27a0db7-0ad9-4556-b402-dedda3af71d1 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:29.691562 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a27a0db7-0ad9-4556-b402-dedda3af71d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:29.791184 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1014/2027] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:29.803937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b0c7a-52d5-4eab-8d84-786d96ec0a67 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:29.804230 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b0c7a-52d5-4eab-8d84-786d96ec0a67 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:29.804513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b0c7a-52d5-4eab-8d84-786d96ec0a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:29.829118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8d2b0c7a-52d5-4eab-8d84-786d96ec0a67 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:29.830388 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8d2b0c7a-52d5-4eab-8d84-786d96ec0a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:29.840021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1014/2028] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:29.845188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a0db7-0ad9-4556-b402-dedda3af71d1 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:29.862772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a0db7-0ad9-4556-b402-dedda3af71d1 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:29.870504 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52393c8-3798-4fec-8c59-1c23be9789dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:29.871520 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52393c8-3798-4fec-8c59-1c23be9789dd admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:29.871846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52393c8-3798-4fec-8c59-1c23be9789dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:29.872514 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52393c8-3798-4fec-8c59-1c23be9789dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:29.873701 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52393c8-3798-4fec-8c59-1c23be9789dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:45:29.874246 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52393c8-3798-4fec-8c59-1c23be9789dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:29.877325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52393c8-3798-4fec-8c59-1c23be9789dd admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:29.899199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1015/2029] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:45:29.904552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b0c7a-52d5-4eab-8d84-786d96ec0a67 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:29.914656 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b0c7a-52d5-4eab-8d84-786d96ec0a67 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:29.919805 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d632885-3c9b-4f1b-8908-1f27f2bb2eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:29.920586 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d632885-3c9b-4f1b-8908-1f27f2bb2eaf admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:29.921008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d632885-3c9b-4f1b-8908-1f27f2bb2eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:29.921280 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d632885-3c9b-4f1b-8908-1f27f2bb2eaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:29.921945 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d632885-3c9b-4f1b-8908-1f27f2bb2eaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:29.923959 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d632885-3c9b-4f1b-8908-1f27f2bb2eaf admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:29.957004 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1015/2030] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:29 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:29.962138 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52393c8-3798-4fec-8c59-1c23be9789dd admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:29.977509 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52393c8-3798-4fec-8c59-1c23be9789dd admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:30.045344 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9599014-8bb3-4c5d-aa9f-c6a96a5af8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:30.046068 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9599014-8bb3-4c5d-aa9f-c6a96a5af8e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:30.046351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9599014-8bb3-4c5d-aa9f-c6a96a5af8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:30.046631 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9599014-8bb3-4c5d-aa9f-c6a96a5af8e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:30.047283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9599014-8bb3-4c5d-aa9f-c6a96a5af8e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:30.049318 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9599014-8bb3-4c5d-aa9f-c6a96a5af8e8 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:30.078083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1016/2031] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:29 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:30.080038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d632885-3c9b-4f1b-8908-1f27f2bb2eaf admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:30.094249 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d632885-3c9b-4f1b-8908-1f27f2bb2eaf admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:30.102863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d94d4d-4ff8-43be-ad78-1ecfdc3132bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:30.104134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d94d4d-4ff8-43be-ad78-1ecfdc3132bf admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:30.104612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d94d4d-4ff8-43be-ad78-1ecfdc3132bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:30.105076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d94d4d-4ff8-43be-ad78-1ecfdc3132bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:30.106128 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d94d4d-4ff8-43be-ad78-1ecfdc3132bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:30.109320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d94d4d-4ff8-43be-ad78-1ecfdc3132bf admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:30.130441 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1016/2032] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:30.132050 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9599014-8bb3-4c5d-aa9f-c6a96a5af8e8 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:30.151599 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9599014-8bb3-4c5d-aa9f-c6a96a5af8e8 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:30.159047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f90ef-252c-4904-8b5e-26793e7e081f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:30.160019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f90ef-252c-4904-8b5e-26793e7e081f admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:30.160464 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f90ef-252c-4904-8b5e-26793e7e081f admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:30.160834 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f90ef-252c-4904-8b5e-26793e7e081f admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:30.161632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f90ef-252c-4904-8b5e-26793e7e081f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:30.174320 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f90ef-252c-4904-8b5e-26793e7e081f admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:30.176024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:30.189109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d94d4d-4ff8-43be-ad78-1ecfdc3132bf admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:30.191228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1017/2033] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:30.204343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d94d4d-4ff8-43be-ad78-1ecfdc3132bf admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:30.210105 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095046d-4336-4664-9b08-c503e59334dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:30.210801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095046d-4336-4664-9b08-c503e59334dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:30.211078 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095046d-4336-4664-9b08-c503e59334dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:30.211351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095046d-4336-4664-9b08-c503e59334dd admin admin] PATH_INFO: `/v3/projects/66b9ecbd3cf846188dd390a21ea1ce54/users/68e9ef76a323421cbd4ab101c00faed5/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:30.230322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095046d-4336-4664-9b08-c503e59334dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b9ecbd3cf846188dd390a21ea1ce54, user_id=68e9ef76a323421cbd4ab101c00faed5, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:30.233393 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095046d-4336-4664-9b08-c503e59334dd admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:30.250657 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1017/2034] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:30 2022] PUT /identity/v3/projects/66b9ecbd3cf846188dd390a21ea1ce54/users/68e9ef76a323421cbd4ab101c00faed5/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:45:30.270370 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4b3c6-e561-40f1-aefd-4ba8b9425ca7 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:30.270703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4b3c6-e561-40f1-aefd-4ba8b9425ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:30.271180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4b3c6-e561-40f1-aefd-4ba8b9425ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:30.326596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-d6b4b3c6-e561-40f1-aefd-4ba8b9425ca7 None None] MFA Rules not processed for user `68e9ef76a323421cbd4ab101c00faed5`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:30.328787 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d6b4b3c6-e561-40f1-aefd-4ba8b9425ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:30.430965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1018/2035] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:30.443399 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095046d-4336-4664-9b08-c503e59334dd admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:30.445454 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f095046d-4336-4664-9b08-c503e59334dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:30.519970 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:30.548011 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095046d-4336-4664-9b08-c503e59334dd admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:30.554692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf79ebb-f31e-4299-a106-fbb7226b8208 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69de9368993f402cb6325e5022e064d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:30.555705 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf79ebb-f31e-4299-a106-fbb7226b8208 service glance] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:30.556216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf79ebb-f31e-4299-a106-fbb7226b8208 service glance] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:30.556749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf79ebb-f31e-4299-a106-fbb7226b8208 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:30.572456 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf79ebb-f31e-4299-a106-fbb7226b8208 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:30.575064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf79ebb-f31e-4299-a106-fbb7226b8208 service glance] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:30.591222 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1018/2036] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:35.421772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4b3c6-e561-40f1-aefd-4ba8b9425ca7 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:35.423966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d6b4b3c6-e561-40f1-aefd-4ba8b9425ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:35.525161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4b3c6-e561-40f1-aefd-4ba8b9425ca7 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:35.531540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdd999-b621-423e-bf89-6ffc8a9052ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:35.532248 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdd999-b621-423e-bf89-6ffc8a9052ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:35.532530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdd999-b621-423e-bf89-6ffc8a9052ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:35.532803 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdd999-b621-423e-bf89-6ffc8a9052ca admin admin] PATH_INFO: `/v3/users/3223d157076a42689adf85c68228bda4` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:35.534084 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdd999-b621-423e-bf89-6ffc8a9052ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=3223d157076a42689adf85c68228bda4)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:35.536084 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdd999-b621-423e-bf89-6ffc8a9052ca admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:35.611072 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-17bdd999-b621-423e-bf89-6ffc8a9052ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3223d157076a42689adf85c68228bda4'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:35.612537 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-17bdd999-b621-423e-bf89-6ffc8a9052ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3223d157076a42689adf85c68228bda4'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:35.613630 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-17bdd999-b621-423e-bf89-6ffc8a9052ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3223d157076a42689adf85c68228bda4'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:35.626043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-17bdd999-b621-423e-bf89-6ffc8a9052ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3223d157076a42689adf85c68228bda4'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:35.649682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-17bdd999-b621-423e-bf89-6ffc8a9052ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3223d157076a42689adf85c68228bda4'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:35.666521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1019/2037] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:35 2022] DELETE /identity/v3/users/3223d157076a42689adf85c68228bda4 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:35.673707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72699ce-ce06-46de-bd0a-e6c13393e3e3 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:35.674878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72699ce-ce06-46de-bd0a-e6c13393e3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:35.675302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72699ce-ce06-46de-bd0a-e6c13393e3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:35.699736 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d72699ce-ce06-46de-bd0a-e6c13393e3e3 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:35.701436 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d72699ce-ce06-46de-bd0a-e6c13393e3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:35.852073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1019/2038] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:35.859840 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdd999-b621-423e-bf89-6ffc8a9052ca admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:35.861634 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-17bdd999-b621-423e-bf89-6ffc8a9052ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:35.875635 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:35.975932 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdd999-b621-423e-bf89-6ffc8a9052ca admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:35.982550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aae7c4-9a41-4530-a671-7683217e1638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:35.983569 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aae7c4-9a41-4530-a671-7683217e1638 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:35.983971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aae7c4-9a41-4530-a671-7683217e1638 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:35.984849 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aae7c4-9a41-4530-a671-7683217e1638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:36.003419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aae7c4-9a41-4530-a671-7683217e1638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:36.004934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aae7c4-9a41-4530-a671-7683217e1638 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:36.027169 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1020/2039] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:36.419969 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72699ce-ce06-46de-bd0a-e6c13393e3e3 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:36.421596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d72699ce-ce06-46de-bd0a-e6c13393e3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:36.455498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72699ce-ce06-46de-bd0a-e6c13393e3e3 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:36.461875 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:36.462874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:36.463362 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:36.463986 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7 admin admin] PATH_INFO: `/v3/projects/1a7e365ff2ed4dad860ef2388da58cdc` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:36.465548 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a7e365ff2ed4dad860ef2388da58cdc)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:36.471124 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:36.572030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a7e365ff2ed4dad860ef2388da58cdc', 'request_id': 'req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:36.595538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a7e365ff2ed4dad860ef2388da58cdc', 'request_id': 'req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:36.613532 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a7e365ff2ed4dad860ef2388da58cdc', 'request_id': 'req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:36.632930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7 admin admin] The token cache is being invalidate because project 1a7e365ff2ed4dad860ef2388da58cdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:36.633449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:36.635079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:36.640181 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1020/2040] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:36 2022] DELETE /identity/v3/projects/1a7e365ff2ed4dad860ef2388da58cdc => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:36.644788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aae7c4-9a41-4530-a671-7683217e1638 service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:36.658083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aae7c4-9a41-4530-a671-7683217e1638 service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:36.784228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7905d5-055b-4b42-92b2-760c7dca1c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:36.785346 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7905d5-055b-4b42-92b2-760c7dca1c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:36.785800 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7905d5-055b-4b42-92b2-760c7dca1c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:36.786183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7905d5-055b-4b42-92b2-760c7dca1c12 admin admin] PATH_INFO: `/v3/users/97d86e2016f444288f8e96a4f74cf165` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:36.787700 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7905d5-055b-4b42-92b2-760c7dca1c12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97d86e2016f444288f8e96a4f74cf165)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:36.790170 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7905d5-055b-4b42-92b2-760c7dca1c12 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:36.866235 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-9f7905d5-055b-4b42-92b2-760c7dca1c12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97d86e2016f444288f8e96a4f74cf165'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:36.867118 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-9f7905d5-055b-4b42-92b2-760c7dca1c12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97d86e2016f444288f8e96a4f74cf165'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:36.867918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-9f7905d5-055b-4b42-92b2-760c7dca1c12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97d86e2016f444288f8e96a4f74cf165'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:36.878483 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-9f7905d5-055b-4b42-92b2-760c7dca1c12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97d86e2016f444288f8e96a4f74cf165'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:36.895401 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-9f7905d5-055b-4b42-92b2-760c7dca1c12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97d86e2016f444288f8e96a4f74cf165'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:36.913368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1021/2041] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:36 2022] DELETE /identity/v3/users/97d86e2016f444288f8e96a4f74cf165 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:37.431469 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:37.435418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:37.468753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b5bb3-dd45-4ac1-8c2d-f7ea6bae46f7 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:37.732544 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e03ae-823a-4248-a9d6-c6922b655d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:37.735104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e03ae-823a-4248-a9d6-c6922b655d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:37.737438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e03ae-823a-4248-a9d6-c6922b655d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:37.738055 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e03ae-823a-4248-a9d6-c6922b655d50 admin admin] PATH_INFO: `/v3/projects/d0c450b38f424605b47db03f5285fc7c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:37.740514 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e03ae-823a-4248-a9d6-c6922b655d50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0c450b38f424605b47db03f5285fc7c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:37.762138 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e03ae-823a-4248-a9d6-c6922b655d50 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:37.783593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:37.921397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d29e03ae-823a-4248-a9d6-c6922b655d50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0c450b38f424605b47db03f5285fc7c', 'request_id': 'req-d29e03ae-823a-4248-a9d6-c6922b655d50'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:37.965250 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d29e03ae-823a-4248-a9d6-c6922b655d50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0c450b38f424605b47db03f5285fc7c', 'request_id': 'req-d29e03ae-823a-4248-a9d6-c6922b655d50'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:37.991077 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d29e03ae-823a-4248-a9d6-c6922b655d50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0c450b38f424605b47db03f5285fc7c', 'request_id': 'req-d29e03ae-823a-4248-a9d6-c6922b655d50'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:38.047765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d29e03ae-823a-4248-a9d6-c6922b655d50 admin admin] The token cache is being invalidate because project d0c450b38f424605b47db03f5285fc7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:38.050450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d29e03ae-823a-4248-a9d6-c6922b655d50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:38.052788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d29e03ae-823a-4248-a9d6-c6922b655d50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:38.059551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1021/2042] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:37 2022] DELETE /identity/v3/projects/d0c450b38f424605b47db03f5285fc7c => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:38.064104 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7905d5-055b-4b42-92b2-760c7dca1c12 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:38.066379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-9f7905d5-055b-4b42-92b2-760c7dca1c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:38.093750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7905d5-055b-4b42-92b2-760c7dca1c12 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:38.138672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:38.201410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf053a-5caf-409a-ab0b-1a793f69707c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:38.202736 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf053a-5caf-409a-ab0b-1a793f69707c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:38.204053 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf053a-5caf-409a-ab0b-1a793f69707c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:38.204667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf053a-5caf-409a-ab0b-1a793f69707c admin admin] PATH_INFO: `/v3/users/f946f8b5a3144be4bcae54c0aa79a6b2` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:38.206524 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf053a-5caf-409a-ab0b-1a793f69707c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f946f8b5a3144be4bcae54c0aa79a6b2)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:38.210446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf053a-5caf-409a-ab0b-1a793f69707c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:38.322159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eabf053a-5caf-409a-ab0b-1a793f69707c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f946f8b5a3144be4bcae54c0aa79a6b2'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:38.323418 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eabf053a-5caf-409a-ab0b-1a793f69707c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f946f8b5a3144be4bcae54c0aa79a6b2'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:38.324375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eabf053a-5caf-409a-ab0b-1a793f69707c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f946f8b5a3144be4bcae54c0aa79a6b2'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:38.340754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eabf053a-5caf-409a-ab0b-1a793f69707c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f946f8b5a3144be4bcae54c0aa79a6b2'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:38.365776 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eabf053a-5caf-409a-ab0b-1a793f69707c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f946f8b5a3144be4bcae54c0aa79a6b2'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:38.390929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1022/2043] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:38 2022] DELETE /identity/v3/users/f946f8b5a3144be4bcae54c0aa79a6b2 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:38.877074 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e03ae-823a-4248-a9d6-c6922b655d50 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:38.879323 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d29e03ae-823a-4248-a9d6-c6922b655d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:38.921302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e03ae-823a-4248-a9d6-c6922b655d50 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:39.002399 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592d664-a868-4481-a225-493ed9c76f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:39.003274 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592d664-a868-4481-a225-493ed9c76f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:39.003563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592d664-a868-4481-a225-493ed9c76f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:39.004578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592d664-a868-4481-a225-493ed9c76f14 admin admin] PATH_INFO: `/v3/projects/78ec4b688b1147f3b7fd39dad9357ab0` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:39.006079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592d664-a868-4481-a225-493ed9c76f14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ec4b688b1147f3b7fd39dad9357ab0)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:39.008163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592d664-a868-4481-a225-493ed9c76f14 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:39.066035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7592d664-a868-4481-a225-493ed9c76f14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ec4b688b1147f3b7fd39dad9357ab0', 'request_id': 'req-7592d664-a868-4481-a225-493ed9c76f14'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:39.074640 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7592d664-a868-4481-a225-493ed9c76f14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ec4b688b1147f3b7fd39dad9357ab0', 'request_id': 'req-7592d664-a868-4481-a225-493ed9c76f14'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:39.084074 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7592d664-a868-4481-a225-493ed9c76f14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ec4b688b1147f3b7fd39dad9357ab0', 'request_id': 'req-7592d664-a868-4481-a225-493ed9c76f14'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:39.098176 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7592d664-a868-4481-a225-493ed9c76f14 admin admin] The token cache is being invalidate because project 78ec4b688b1147f3b7fd39dad9357ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:39.098503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7592d664-a868-4481-a225-493ed9c76f14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:39.099215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7592d664-a868-4481-a225-493ed9c76f14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:39.100916 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1022/2044] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:38 2022] DELETE /identity/v3/projects/78ec4b688b1147f3b7fd39dad9357ab0 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:39.106625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4804fc-68f1-4d66-b190-64a8f1ecf7be None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:39.107242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4804fc-68f1-4d66-b190-64a8f1ecf7be None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:39.107534 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4804fc-68f1-4d66-b190-64a8f1ecf7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:39.133190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-9b4804fc-68f1-4d66-b190-64a8f1ecf7be None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:39.134767 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-9b4804fc-68f1-4d66-b190-64a8f1ecf7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:39.227210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1023/2045] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:39.236230 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719381b-52c6-4caa-83c0-d72543df1fa8 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:39.236612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719381b-52c6-4caa-83c0-d72543df1fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:39.236984 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719381b-52c6-4caa-83c0-d72543df1fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:39.260868 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f719381b-52c6-4caa-83c0-d72543df1fa8 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:39.262244 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f719381b-52c6-4caa-83c0-d72543df1fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:39.275041 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1023/2046] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:39.281243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4804fc-68f1-4d66-b190-64a8f1ecf7be None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:39.293204 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4804fc-68f1-4d66-b190-64a8f1ecf7be None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:39.299340 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61c567-7c51-4a98-87f1-8ac5a41ccaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:39.306485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61c567-7c51-4a98-87f1-8ac5a41ccaf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:39.306485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61c567-7c51-4a98-87f1-8ac5a41ccaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:39.306485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61c567-7c51-4a98-87f1-8ac5a41ccaf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:39.306485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61c567-7c51-4a98-87f1-8ac5a41ccaf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:45:39.306485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61c567-7c51-4a98-87f1-8ac5a41ccaf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:39.306485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61c567-7c51-4a98-87f1-8ac5a41ccaf3 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:39.317750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1024/2047] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:45:39.322753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719381b-52c6-4caa-83c0-d72543df1fa8 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:39.334868 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719381b-52c6-4caa-83c0-d72543df1fa8 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:39.341370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec7036-f528-43f1-a784-c16e8486c471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:39.342222 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec7036-f528-43f1-a784-c16e8486c471 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:39.342618 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec7036-f528-43f1-a784-c16e8486c471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:39.343060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec7036-f528-43f1-a784-c16e8486c471 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:39.343946 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec7036-f528-43f1-a784-c16e8486c471 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:39.346729 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec7036-f528-43f1-a784-c16e8486c471 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:39.369213 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:39.397251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1024/2048] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:39 2022] POST /identity/v3/projects => generated 369 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:39.398526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61c567-7c51-4a98-87f1-8ac5a41ccaf3 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:39.413269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61c567-7c51-4a98-87f1-8ac5a41ccaf3 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:39.472251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:39.509477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf16589-7da3-4666-bcfa-5f49cdc13462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:39.511227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf16589-7da3-4666-bcfa-5f49cdc13462 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:39.512124 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf16589-7da3-4666-bcfa-5f49cdc13462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:39.513049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf16589-7da3-4666-bcfa-5f49cdc13462 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:39.514559 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf16589-7da3-4666-bcfa-5f49cdc13462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:39.521754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf16589-7da3-4666-bcfa-5f49cdc13462 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:39.558763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1025/2049] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:39 2022] POST /identity/v3/users => generated 285 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:39.560425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec7036-f528-43f1-a784-c16e8486c471 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:39.579224 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec7036-f528-43f1-a784-c16e8486c471 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:39.589243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b902d5-c870-445c-ac1e-8930c393012a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:39.589940 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b902d5-c870-445c-ac1e-8930c393012a admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:39.590219 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b902d5-c870-445c-ac1e-8930c393012a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:39.590534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b902d5-c870-445c-ac1e-8930c393012a admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:39.591460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b902d5-c870-445c-ac1e-8930c393012a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:39.593498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b902d5-c870-445c-ac1e-8930c393012a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:39.611686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1025/2050] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:39.616352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf16589-7da3-4666-bcfa-5f49cdc13462 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:39.633887 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf16589-7da3-4666-bcfa-5f49cdc13462 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:39.647065 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cd2ce-1fd5-4e61-9d27-64357a5d6762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:39.648354 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cd2ce-1fd5-4e61-9d27-64357a5d6762 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:39.648836 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cd2ce-1fd5-4e61-9d27-64357a5d6762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:39.649317 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cd2ce-1fd5-4e61-9d27-64357a5d6762 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:39.650550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67cd2ce-1fd5-4e61-9d27-64357a5d6762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:39.653792 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67cd2ce-1fd5-4e61-9d27-64357a5d6762 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:39.675979 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1026/2051] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:39.681209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b902d5-c870-445c-ac1e-8930c393012a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:39.702934 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b902d5-c870-445c-ac1e-8930c393012a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:39.713841 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f1074-9b92-43d7-9d94-38b591f25f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:39.715270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f1074-9b92-43d7-9d94-38b591f25f80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:39.715794 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f1074-9b92-43d7-9d94-38b591f25f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:39.716354 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f1074-9b92-43d7-9d94-38b591f25f80 admin admin] PATH_INFO: `/v3/projects/13cd13ed995243b4ae4a97c10ddeff01/users/075e43152aaf44d0b47d2d397fcda602/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:39.752488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f1074-9b92-43d7-9d94-38b591f25f80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13cd13ed995243b4ae4a97c10ddeff01, user_id=075e43152aaf44d0b47d2d397fcda602, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:39.758101 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f1074-9b92-43d7-9d94-38b591f25f80 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:39.784352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1026/2052] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:39 2022] PUT /identity/v3/projects/13cd13ed995243b4ae4a97c10ddeff01/users/075e43152aaf44d0b47d2d397fcda602/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:39.796607 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df277d05-811f-4541-b990-480a8fc40ea3 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:39.797245 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df277d05-811f-4541-b990-480a8fc40ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:39.797679 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df277d05-811f-4541-b990-480a8fc40ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:39.844162 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-df277d05-811f-4541-b990-480a8fc40ea3 None None] MFA Rules not processed for user `075e43152aaf44d0b47d2d397fcda602`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:39.845342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-df277d05-811f-4541-b990-480a8fc40ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:39.956210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1027/2053] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:45:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:39.969159 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f1074-9b92-43d7-9d94-38b591f25f80 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:39.971119 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d30f1074-9b92-43d7-9d94-38b591f25f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:40.085694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f1074-9b92-43d7-9d94-38b591f25f80 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:40.093295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006bc6a-3ee5-4073-8bd6-644d50c04f5c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69de9368993f402cb6325e5022e064d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:40.094448 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006bc6a-3ee5-4073-8bd6-644d50c04f5c service glance] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:40.094985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006bc6a-3ee5-4073-8bd6-644d50c04f5c service glance] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:40.095914 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006bc6a-3ee5-4073-8bd6-644d50c04f5c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:40.132073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006bc6a-3ee5-4073-8bd6-644d50c04f5c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:40.136070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006bc6a-3ee5-4073-8bd6-644d50c04f5c service glance] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:40.162872 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1027/2054] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:49.951429 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=49,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:50.146461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:46:08.754290 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df277d05-811f-4541-b990-480a8fc40ea3 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:08.755796 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-df277d05-811f-4541-b990-480a8fc40ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:08.847815 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df277d05-811f-4541-b990-480a8fc40ea3 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:08.853026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87dce53-20c2-48a6-a721-affa3face7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:08.853781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87dce53-20c2-48a6-a721-affa3face7ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:08.854165 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87dce53-20c2-48a6-a721-affa3face7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:08.854381 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87dce53-20c2-48a6-a721-affa3face7ae admin admin] PATH_INFO: `/v3/users/036bcef8c5604c5b8e26420c3bb310ac` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:08.855679 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87dce53-20c2-48a6-a721-affa3face7ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=036bcef8c5604c5b8e26420c3bb310ac)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:08.858349 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87dce53-20c2-48a6-a721-affa3face7ae admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:08.938134 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-c87dce53-20c2-48a6-a721-affa3face7ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '036bcef8c5604c5b8e26420c3bb310ac'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:08.939254 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-c87dce53-20c2-48a6-a721-affa3face7ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '036bcef8c5604c5b8e26420c3bb310ac'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:08.940288 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-c87dce53-20c2-48a6-a721-affa3face7ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '036bcef8c5604c5b8e26420c3bb310ac'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:08.957625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-c87dce53-20c2-48a6-a721-affa3face7ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '036bcef8c5604c5b8e26420c3bb310ac'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:08.988440 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-c87dce53-20c2-48a6-a721-affa3face7ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '036bcef8c5604c5b8e26420c3bb310ac'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:09.017301 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1028/2055] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:46:08 2022] DELETE /identity/v3/users/036bcef8c5604c5b8e26420c3bb310ac => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:46:09.023222 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4f328-b120-4722-af3d-7434bb94d9f3 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:09.024170 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4f328-b120-4722-af3d-7434bb94d9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:09.024170 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4f328-b120-4722-af3d-7434bb94d9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:09.048071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0cf4f328-b120-4722-af3d-7434bb94d9f3 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:46:09.049325 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0cf4f328-b120-4722-af3d-7434bb94d9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:09.145984 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1028/2056] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:46:09.157619 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87dce53-20c2-48a6-a721-affa3face7ae admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:09.160124 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c87dce53-20c2-48a6-a721-affa3face7ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:09.262935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87dce53-20c2-48a6-a721-affa3face7ae admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:09.268141 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a62c9-7389-416e-a1b5-ba2bd83adadd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:09.268443 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a62c9-7389-416e-a1b5-ba2bd83adadd service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:09.268744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a62c9-7389-416e-a1b5-ba2bd83adadd service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:09.268977 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a62c9-7389-416e-a1b5-ba2bd83adadd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:09.279974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a62c9-7389-416e-a1b5-ba2bd83adadd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:09.281743 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a62c9-7389-416e-a1b5-ba2bd83adadd service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:09.298667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1029/2057] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:46:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:46:09.726641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4f328-b120-4722-af3d-7434bb94d9f3 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:09.728929 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0cf4f328-b120-4722-af3d-7434bb94d9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:09.751936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4f328-b120-4722-af3d-7434bb94d9f3 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:09.760112 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c477f-7fb5-4ff8-848c-b420a78b4113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:09.761203 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c477f-7fb5-4ff8-848c-b420a78b4113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:09.761661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c477f-7fb5-4ff8-848c-b420a78b4113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:09.762226 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c477f-7fb5-4ff8-848c-b420a78b4113 admin admin] PATH_INFO: `/v3/projects/a641327daaa04f398b094dab45352b30` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:09.764432 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c477f-7fb5-4ff8-848c-b420a78b4113 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a641327daaa04f398b094dab45352b30)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:09.768256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c477f-7fb5-4ff8-848c-b420a78b4113 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:09.865990 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d32c477f-7fb5-4ff8-848c-b420a78b4113 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a641327daaa04f398b094dab45352b30', 'request_id': 'req-d32c477f-7fb5-4ff8-848c-b420a78b4113'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:09.881735 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d32c477f-7fb5-4ff8-848c-b420a78b4113 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a641327daaa04f398b094dab45352b30', 'request_id': 'req-d32c477f-7fb5-4ff8-848c-b420a78b4113'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:09.910289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d32c477f-7fb5-4ff8-848c-b420a78b4113 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a641327daaa04f398b094dab45352b30', 'request_id': 'req-d32c477f-7fb5-4ff8-848c-b420a78b4113'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:09.924701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d32c477f-7fb5-4ff8-848c-b420a78b4113 admin admin] The token cache is being invalidate because project a641327daaa04f398b094dab45352b30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:46:09.925949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d32c477f-7fb5-4ff8-848c-b420a78b4113 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:09.926438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d32c477f-7fb5-4ff8-848c-b420a78b4113 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:09.932890 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1029/2058] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:46:09 2022] DELETE /identity/v3/projects/a641327daaa04f398b094dab45352b30 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:46:09.935908 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963abad6-b437-4134-9f9f-1792295bf335 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:09.936753 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963abad6-b437-4134-9f9f-1792295bf335 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:09.936985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963abad6-b437-4134-9f9f-1792295bf335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:09.978937 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-963abad6-b437-4134-9f9f-1792295bf335 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:46:09.980948 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-963abad6-b437-4134-9f9f-1792295bf335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:10.013033 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:46:10.095700 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1030/2059] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:46:10.099673 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5c57c-4382-41c0-9088-1edbb0373e3d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:10.101132 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5c57c-4382-41c0-9088-1edbb0373e3d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:10.101884 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5c57c-4382-41c0-9088-1edbb0373e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:10.136338 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0bf5c57c-4382-41c0-9088-1edbb0373e3d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:46:10.137733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0bf5c57c-4382-41c0-9088-1edbb0373e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:10.146783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1030/2060] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:46:10.153693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963abad6-b437-4134-9f9f-1792295bf335 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:10.167045 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963abad6-b437-4134-9f9f-1792295bf335 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:10.174512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b438c3-6635-4d27-98b4-2949d2703b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:10.175578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b438c3-6635-4d27-98b4-2949d2703b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:10.175578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b438c3-6635-4d27-98b4-2949d2703b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:10.175885 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b438c3-6635-4d27-98b4-2949d2703b0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:10.176526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b438c3-6635-4d27-98b4-2949d2703b0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:46:10.176971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b438c3-6635-4d27-98b4-2949d2703b0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:10.178794 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b438c3-6635-4d27-98b4-2949d2703b0d admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:10.197155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1031/2061] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:46:10.201841 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5c57c-4382-41c0-9088-1edbb0373e3d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:10.219744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5c57c-4382-41c0-9088-1edbb0373e3d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:10.226424 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32791081-ecd8-4178-a9e6-dd726901dd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:10.227374 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32791081-ecd8-4178-a9e6-dd726901dd90 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:10.227895 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32791081-ecd8-4178-a9e6-dd726901dd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:10.228898 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32791081-ecd8-4178-a9e6-dd726901dd90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:10.230333 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32791081-ecd8-4178-a9e6-dd726901dd90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:10.233278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32791081-ecd8-4178-a9e6-dd726901dd90 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:10.305087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1031/2062] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:46:10 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:46:10.311071 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b438c3-6635-4d27-98b4-2949d2703b0d admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:10.330093 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b438c3-6635-4d27-98b4-2949d2703b0d admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:10.452618 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1164e7-a720-4f7c-b9c2-a8d6fea613d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:10.453265 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1164e7-a720-4f7c-b9c2-a8d6fea613d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:10.453561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1164e7-a720-4f7c-b9c2-a8d6fea613d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:10.453907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1164e7-a720-4f7c-b9c2-a8d6fea613d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:10.454620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1164e7-a720-4f7c-b9c2-a8d6fea613d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:10.456953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1164e7-a720-4f7c-b9c2-a8d6fea613d8 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:10.492073 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1032/2063] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:46:10 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:46:10.496303 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32791081-ecd8-4178-a9e6-dd726901dd90 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:10.511765 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32791081-ecd8-4178-a9e6-dd726901dd90 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:10.519374 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd66e85-5614-4ae9-a98e-31ebb4829552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:10.520415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd66e85-5614-4ae9-a98e-31ebb4829552 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:10.520865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd66e85-5614-4ae9-a98e-31ebb4829552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:10.521296 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd66e85-5614-4ae9-a98e-31ebb4829552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:10.522414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd66e85-5614-4ae9-a98e-31ebb4829552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:10.525444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd66e85-5614-4ae9-a98e-31ebb4829552 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:10.545771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1032/2064] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:46:10.549358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1164e7-a720-4f7c-b9c2-a8d6fea613d8 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:10.561366 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1164e7-a720-4f7c-b9c2-a8d6fea613d8 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:10.566495 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e704019-99ea-41e3-8da9-7b206320bfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:10.567175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e704019-99ea-41e3-8da9-7b206320bfb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:10.567450 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e704019-99ea-41e3-8da9-7b206320bfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:10.567714 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e704019-99ea-41e3-8da9-7b206320bfb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:10.568411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e704019-99ea-41e3-8da9-7b206320bfb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:10.570066 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e704019-99ea-41e3-8da9-7b206320bfb8 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:10.596628 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1033/2065] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:46:10.604772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd66e85-5614-4ae9-a98e-31ebb4829552 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:10.615819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd66e85-5614-4ae9-a98e-31ebb4829552 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:10.621135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d6b3c-4bf4-4757-9d02-afef4cabea2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:10.622634 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d6b3c-4bf4-4757-9d02-afef4cabea2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:10.629198 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d6b3c-4bf4-4757-9d02-afef4cabea2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:10.629198 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d6b3c-4bf4-4757-9d02-afef4cabea2f admin admin] PATH_INFO: `/v3/projects/39e7bfd5fca640109a055032971dfcb7/users/5e260fff0fab4cd0979047116a1e3e32/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:10.650497 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d6b3c-4bf4-4757-9d02-afef4cabea2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e7bfd5fca640109a055032971dfcb7, user_id=5e260fff0fab4cd0979047116a1e3e32, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:10.657783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d6b3c-4bf4-4757-9d02-afef4cabea2f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:10.678580 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1033/2066] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:46:10 2022] PUT /identity/v3/projects/39e7bfd5fca640109a055032971dfcb7/users/5e260fff0fab4cd0979047116a1e3e32/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:46:10.691408 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc036ba-41f5-404e-abc0-5cc96edb88b0 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:10.691850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc036ba-41f5-404e-abc0-5cc96edb88b0 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:10.692176 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc036ba-41f5-404e-abc0-5cc96edb88b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:10.739522 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-7bc036ba-41f5-404e-abc0-5cc96edb88b0 None None] MFA Rules not processed for user `5e260fff0fab4cd0979047116a1e3e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:46:10.741748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7bc036ba-41f5-404e-abc0-5cc96edb88b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:10.873205 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1034/2067] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:46:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:46:10.892590 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d6b3c-4bf4-4757-9d02-afef4cabea2f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:10.895225 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cf7d6b3c-4bf4-4757-9d02-afef4cabea2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:10.925457 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:46:10.998207 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d6b3c-4bf4-4757-9d02-afef4cabea2f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:11.003269 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b78872-610f-4b06-ae69-9fabf27d235c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:11.004143 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b78872-610f-4b06-ae69-9fabf27d235c service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:11.004417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b78872-610f-4b06-ae69-9fabf27d235c service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:11.004768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b78872-610f-4b06-ae69-9fabf27d235c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:11.021406 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b78872-610f-4b06-ae69-9fabf27d235c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:11.022880 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b78872-610f-4b06-ae69-9fabf27d235c service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:11.038379 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1034/2068] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:46:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:46:13.681918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961d041-8632-4d23-a73e-98e4707edeb3 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:13.683004 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961d041-8632-4d23-a73e-98e4707edeb3 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:13.683004 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961d041-8632-4d23-a73e-98e4707edeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:13.705920 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-b961d041-8632-4d23-a73e-98e4707edeb3 None None] MFA Rules not processed for user `5e260fff0fab4cd0979047116a1e3e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:46:13.707184 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b961d041-8632-4d23-a73e-98e4707edeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:13.714976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1035/2069] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:46:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:46:18.591745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b78872-610f-4b06-ae69-9fabf27d235c service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:18.594301 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-14b78872-610f-4b06-ae69-9fabf27d235c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:18.715071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b78872-610f-4b06-ae69-9fabf27d235c service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:18.722233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44acf2a-977a-46d7-9db6-69dc5673c008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:18.722923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44acf2a-977a-46d7-9db6-69dc5673c008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:18.723205 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44acf2a-977a-46d7-9db6-69dc5673c008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:18.723479 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44acf2a-977a-46d7-9db6-69dc5673c008 admin admin] PATH_INFO: `/v3/users/68e9ef76a323421cbd4ab101c00faed5` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:18.725713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44acf2a-977a-46d7-9db6-69dc5673c008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68e9ef76a323421cbd4ab101c00faed5)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:18.729021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44acf2a-977a-46d7-9db6-69dc5673c008 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:18.821889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a44acf2a-977a-46d7-9db6-69dc5673c008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68e9ef76a323421cbd4ab101c00faed5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:18.838600 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a44acf2a-977a-46d7-9db6-69dc5673c008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68e9ef76a323421cbd4ab101c00faed5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:18.867653 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a44acf2a-977a-46d7-9db6-69dc5673c008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68e9ef76a323421cbd4ab101c00faed5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:18.868442 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a44acf2a-977a-46d7-9db6-69dc5673c008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68e9ef76a323421cbd4ab101c00faed5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:18.911374 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a44acf2a-977a-46d7-9db6-69dc5673c008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68e9ef76a323421cbd4ab101c00faed5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:18.932040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1035/2070] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:46:18 2022] DELETE /identity/v3/users/68e9ef76a323421cbd4ab101c00faed5 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:46:18.937479 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83dbf8e-10ed-4602-a706-ecb6f80b53c9 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:18.938283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83dbf8e-10ed-4602-a706-ecb6f80b53c9 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:18.938283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83dbf8e-10ed-4602-a706-ecb6f80b53c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:18.998156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-c83dbf8e-10ed-4602-a706-ecb6f80b53c9 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:46:19.001506 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c83dbf8e-10ed-4602-a706-ecb6f80b53c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:19.012624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:46:19.173312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1036/2071] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:46:19.194871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44acf2a-977a-46d7-9db6-69dc5673c008 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:19.202403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a44acf2a-977a-46d7-9db6-69dc5673c008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:19.316653 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:46:19.335863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44acf2a-977a-46d7-9db6-69dc5673c008 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:19.346903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c4529-af4b-4e70-a635-9d9c959775f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:19.350655 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c4529-af4b-4e70-a635-9d9c959775f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:19.351746 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c4529-af4b-4e70-a635-9d9c959775f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:19.351746 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c4529-af4b-4e70-a635-9d9c959775f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:19.408339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c4529-af4b-4e70-a635-9d9c959775f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:19.411885 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c4529-af4b-4e70-a635-9d9c959775f4 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:19.433787 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1036/2072] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:46:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:46:19.784958 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa978d2a-1fb7-496d-9647-1ba3069cafd0 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:19.786282 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa978d2a-1fb7-496d-9647-1ba3069cafd0 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:19.786282 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa978d2a-1fb7-496d-9647-1ba3069cafd0 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:19.789256 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1037/2073] 10.222.0.25 () {54 vars in 921 bytes} [Tue Aug 30 14:46:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:46:19.808620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81852fe5-39b9-4f31-ab68-ba5739442dc1 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:19.809664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81852fe5-39b9-4f31-ab68-ba5739442dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:19.810087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81852fe5-39b9-4f31-ab68-ba5739442dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:19.864475 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-81852fe5-39b9-4f31-ab68-ba5739442dc1 None None] MFA Rules not processed for user `d7cdd6b1ee9f4e97beb9bb037e446c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:46:19.868733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-81852fe5-39b9-4f31-ab68-ba5739442dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:20.007494 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa978d2a-1fb7-496d-9647-1ba3069cafd0 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:20.010266 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-aa978d2a-1fb7-496d-9647-1ba3069cafd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:20.037717 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa978d2a-1fb7-496d-9647-1ba3069cafd0 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:20.043365 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b274e7-bbcd-45d0-89b1-8e546d3d3824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:20.044373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b274e7-bbcd-45d0-89b1-8e546d3d3824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:20.044807 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b274e7-bbcd-45d0-89b1-8e546d3d3824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:20.046177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b274e7-bbcd-45d0-89b1-8e546d3d3824 admin admin] PATH_INFO: `/v3/users/ee02494e189d4823ab776ea3da9fd2ee` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:20.048604 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b274e7-bbcd-45d0-89b1-8e546d3d3824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee02494e189d4823ab776ea3da9fd2ee)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:20.050919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b274e7-bbcd-45d0-89b1-8e546d3d3824 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:20.142963 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1037/2074] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Aug 30 14:46:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:46:20.156636 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b7b274e7-bbcd-45d0-89b1-8e546d3d3824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee02494e189d4823ab776ea3da9fd2ee'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:20.157446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b7b274e7-bbcd-45d0-89b1-8e546d3d3824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee02494e189d4823ab776ea3da9fd2ee'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:20.158314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b7b274e7-bbcd-45d0-89b1-8e546d3d3824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee02494e189d4823ab776ea3da9fd2ee'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:20.163086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81852fe5-39b9-4f31-ab68-ba5739442dc1 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:20.164661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-81852fe5-39b9-4f31-ab68-ba5739442dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:20.186199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b7b274e7-bbcd-45d0-89b1-8e546d3d3824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee02494e189d4823ab776ea3da9fd2ee'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:20.188498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81852fe5-39b9-4f31-ab68-ba5739442dc1 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:20.195424 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda07fa-6209-4a12-adc7-84ba3852e29c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cdd6b1ee9f4e97beb9bb037e446c8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:20.196240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda07fa-6209-4a12-adc7-84ba3852e29c service placement] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:20.196589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda07fa-6209-4a12-adc7-84ba3852e29c service placement] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:20.197165 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda07fa-6209-4a12-adc7-84ba3852e29c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:20.202099 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3cda07fa-6209-4a12-adc7-84ba3852e29c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:20.221952 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda07fa-6209-4a12-adc7-84ba3852e29c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:20.223731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda07fa-6209-4a12-adc7-84ba3852e29c service placement] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:20.236599 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b7b274e7-bbcd-45d0-89b1-8e546d3d3824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee02494e189d4823ab776ea3da9fd2ee'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:20.238564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1038/2075] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:46:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:46:20.282166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1038/2076] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:46:20 2022] DELETE /identity/v3/users/ee02494e189d4823ab776ea3da9fd2ee => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:46:20.516531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda07fa-6209-4a12-adc7-84ba3852e29c service placement] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:20.519507 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3cda07fa-6209-4a12-adc7-84ba3852e29c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:20.543574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda07fa-6209-4a12-adc7-84ba3852e29c service placement] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:20.644707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc3b40-9466-4807-850b-499af12f0ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:20.645153 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc3b40-9466-4807-850b-499af12f0ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:20.645724 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc3b40-9466-4807-850b-499af12f0ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:20.646135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc3b40-9466-4807-850b-499af12f0ccd admin admin] PATH_INFO: `/v3/projects/7914879d13ad4a589f93b0ce22538d06` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:20.648709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc3b40-9466-4807-850b-499af12f0ccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7914879d13ad4a589f93b0ce22538d06)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:20.656684 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc3b40-9466-4807-850b-499af12f0ccd admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:20.706145 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b274e7-bbcd-45d0-89b1-8e546d3d3824 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:20.711044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b7b274e7-bbcd-45d0-89b1-8e546d3d3824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:20.735490 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b274e7-bbcd-45d0-89b1-8e546d3d3824 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:20.745397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91c406-9f2e-466f-a33e-ee1ced1edc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:20.746668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91c406-9f2e-466f-a33e-ee1ced1edc2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:20.747755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91c406-9f2e-466f-a33e-ee1ced1edc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:20.748614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91c406-9f2e-466f-a33e-ee1ced1edc2d admin admin] PATH_INFO: `/v3/projects/66b9ecbd3cf846188dd390a21ea1ce54` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:20.751295 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91c406-9f2e-466f-a33e-ee1ced1edc2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=66b9ecbd3cf846188dd390a21ea1ce54)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:20.756197 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91c406-9f2e-466f-a33e-ee1ced1edc2d admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:20.776721 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d0cc3b40-9466-4807-850b-499af12f0ccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7914879d13ad4a589f93b0ce22538d06', 'request_id': 'req-d0cc3b40-9466-4807-850b-499af12f0ccd'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:20.788333 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d0cc3b40-9466-4807-850b-499af12f0ccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7914879d13ad4a589f93b0ce22538d06', 'request_id': 'req-d0cc3b40-9466-4807-850b-499af12f0ccd'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:20.797874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d0cc3b40-9466-4807-850b-499af12f0ccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7914879d13ad4a589f93b0ce22538d06', 'request_id': 'req-d0cc3b40-9466-4807-850b-499af12f0ccd'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:20.808298 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:46:20.819105 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d0cc3b40-9466-4807-850b-499af12f0ccd admin admin] The token cache is being invalidate because project 7914879d13ad4a589f93b0ce22538d06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:46:20.820293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d0cc3b40-9466-4807-850b-499af12f0ccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:20.823326 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d0cc3b40-9466-4807-850b-499af12f0ccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:20.826268 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1039/2077] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:46:20 2022] DELETE /identity/v3/projects/7914879d13ad4a589f93b0ce22538d06 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:46:20.832721 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab35c9-f39a-4d95-8487-04485e43463a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:20.833001 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab35c9-f39a-4d95-8487-04485e43463a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:20.833257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab35c9-f39a-4d95-8487-04485e43463a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:20.835309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-3b91c406-9f2e-466f-a33e-ee1ced1edc2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66b9ecbd3cf846188dd390a21ea1ce54', 'request_id': 'req-3b91c406-9f2e-466f-a33e-ee1ced1edc2d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:20.842658 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-3b91c406-9f2e-466f-a33e-ee1ced1edc2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66b9ecbd3cf846188dd390a21ea1ce54', 'request_id': 'req-3b91c406-9f2e-466f-a33e-ee1ced1edc2d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:20.850836 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-3b91c406-9f2e-466f-a33e-ee1ced1edc2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66b9ecbd3cf846188dd390a21ea1ce54', 'request_id': 'req-3b91c406-9f2e-466f-a33e-ee1ced1edc2d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:20.854394 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7bab35c9-f39a-4d95-8487-04485e43463a None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:46:20.856578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7bab35c9-f39a-4d95-8487-04485e43463a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:20.882061 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-3b91c406-9f2e-466f-a33e-ee1ced1edc2d admin admin] The token cache is being invalidate because project 66b9ecbd3cf846188dd390a21ea1ce54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:46:20.882350 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-3b91c406-9f2e-466f-a33e-ee1ced1edc2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:20.883163 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-3b91c406-9f2e-466f-a33e-ee1ced1edc2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:46:20.885843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1039/2078] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:46:20 2022] DELETE /identity/v3/projects/66b9ecbd3cf846188dd390a21ea1ce54 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:46:20.891465 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fd7cd-24c1-4bdf-8899-0b62d234f3cf None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:20.891987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fd7cd-24c1-4bdf-8899-0b62d234f3cf None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:20.892386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fd7cd-24c1-4bdf-8899-0b62d234f3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:20.912640 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:46:20.921236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-937fd7cd-24c1-4bdf-8899-0b62d234f3cf None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:46:20.924180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-937fd7cd-24c1-4bdf-8899-0b62d234f3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:20.942935 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1040/2079] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:46:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:46:20.952135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50869577-6a87-4a13-a75d-52ccff1aeb8c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:20.952461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50869577-6a87-4a13-a75d-52ccff1aeb8c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:20.952802 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50869577-6a87-4a13-a75d-52ccff1aeb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:20.982780 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-50869577-6a87-4a13-a75d-52ccff1aeb8c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:46:20.985748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-50869577-6a87-4a13-a75d-52ccff1aeb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:20.999228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1041/2080] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:46:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:46:21.003786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50869577-6a87-4a13-a75d-52ccff1aeb8c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:21.012659 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1040/2081] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:46:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:46:21.016431 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046b023-224d-48c0-ae18-b15669e019be None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:21.016725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046b023-224d-48c0-ae18-b15669e019be None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:21.017125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046b023-224d-48c0-ae18-b15669e019be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:21.018859 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50869577-6a87-4a13-a75d-52ccff1aeb8c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:21.024613 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96c967-fec4-485d-80e7-af58ee1388bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:21.025418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96c967-fec4-485d-80e7-af58ee1388bd admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:21.025828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96c967-fec4-485d-80e7-af58ee1388bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:21.026211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96c967-fec4-485d-80e7-af58ee1388bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:21.027025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96c967-fec4-485d-80e7-af58ee1388bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:46:21.027854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96c967-fec4-485d-80e7-af58ee1388bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:21.030124 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96c967-fec4-485d-80e7-af58ee1388bd admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:21.039144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-6046b023-224d-48c0-ae18-b15669e019be None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:46:21.040446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6046b023-224d-48c0-ae18-b15669e019be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:21.050036 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1042/2082] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:46:21.050256 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1041/2083] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:46:21.052739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96c967-fec4-485d-80e7-af58ee1388bd admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:21.054467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046b023-224d-48c0-ae18-b15669e019be None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:21.068563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96c967-fec4-485d-80e7-af58ee1388bd admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:21.070386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046b023-224d-48c0-ae18-b15669e019be None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:21.075187 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c8198-0b29-4c85-b0eb-6fae67300efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:21.075897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c8198-0b29-4c85-b0eb-6fae67300efb admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:21.076299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c8198-0b29-4c85-b0eb-6fae67300efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:21.076620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c8198-0b29-4c85-b0eb-6fae67300efb admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:21.077547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c8198-0b29-4c85-b0eb-6fae67300efb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:21.079818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c8198-0b29-4c85-b0eb-6fae67300efb admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:21.079988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3753c22-1370-4a2b-8461-8cc9f49a3b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:21.081186 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3753c22-1370-4a2b-8461-8cc9f49a3b04 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:21.081788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3753c22-1370-4a2b-8461-8cc9f49a3b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:21.082320 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3753c22-1370-4a2b-8461-8cc9f49a3b04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:21.083606 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3753c22-1370-4a2b-8461-8cc9f49a3b04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:46:21.084202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3753c22-1370-4a2b-8461-8cc9f49a3b04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:21.087415 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3753c22-1370-4a2b-8461-8cc9f49a3b04 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:21.125229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1042/2084] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:46:21.127273 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1043/2085] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:46:21 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:46:21.130428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3753c22-1370-4a2b-8461-8cc9f49a3b04 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:21.132918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c8198-0b29-4c85-b0eb-6fae67300efb admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:21.148941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3753c22-1370-4a2b-8461-8cc9f49a3b04 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:21.211851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c8198-0b29-4c85-b0eb-6fae67300efb admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:21.222618 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b26fd2-a2a8-4cff-ab64-370a10129d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:21.223671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b26fd2-a2a8-4cff-ab64-370a10129d0b admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:21.224380 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b26fd2-a2a8-4cff-ab64-370a10129d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:21.224871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b26fd2-a2a8-4cff-ab64-370a10129d0b admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:21.226071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b26fd2-a2a8-4cff-ab64-370a10129d0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:21.229698 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b26fd2-a2a8-4cff-ab64-370a10129d0b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:21.239700 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92712813-2110-40a5-b087-acbe1f602a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:21.240358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92712813-2110-40a5-b087-acbe1f602a73 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:21.240650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92712813-2110-40a5-b087-acbe1f602a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:21.240909 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92712813-2110-40a5-b087-acbe1f602a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:21.241573 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92712813-2110-40a5-b087-acbe1f602a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:21.243901 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92712813-2110-40a5-b087-acbe1f602a73 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:21.260120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1044/2086] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:46:21 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:46:21.262508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b26fd2-a2a8-4cff-ab64-370a10129d0b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:21.274975 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b26fd2-a2a8-4cff-ab64-370a10129d0b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:21.282215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723608b4-debf-47f1-b11f-9579de8c2f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:21.286551 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1043/2087] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:46:21 2022] POST /identity/v3/projects => generated 371 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:46:21.286873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723608b4-debf-47f1-b11f-9579de8c2f3b admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:21.286873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723608b4-debf-47f1-b11f-9579de8c2f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:21.286873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723608b4-debf-47f1-b11f-9579de8c2f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:21.286873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723608b4-debf-47f1-b11f-9579de8c2f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:21.287982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92712813-2110-40a5-b087-acbe1f602a73 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:21.289750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723608b4-debf-47f1-b11f-9579de8c2f3b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:21.310783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1045/2088] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:46:21.315364 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723608b4-debf-47f1-b11f-9579de8c2f3b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:21.332495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723608b4-debf-47f1-b11f-9579de8c2f3b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:21.389416 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92712813-2110-40a5-b087-acbe1f602a73 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:21.394600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64556bef-8f87-4d07-bf38-5c4a8acd4691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:21.395321 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64556bef-8f87-4d07-bf38-5c4a8acd4691 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:21.395599 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64556bef-8f87-4d07-bf38-5c4a8acd4691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:21.395851 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64556bef-8f87-4d07-bf38-5c4a8acd4691 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:21.396560 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64556bef-8f87-4d07-bf38-5c4a8acd4691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:21.398529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64556bef-8f87-4d07-bf38-5c4a8acd4691 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:21.407321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:46:21.431383 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1044/2089] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:46:21 2022] POST /identity/v3/users => generated 286 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:46:21.437248 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64556bef-8f87-4d07-bf38-5c4a8acd4691 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:21.443127 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f427b-8f9a-4d85-88ca-2ec5f6abc138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:21.443956 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f427b-8f9a-4d85-88ca-2ec5f6abc138 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:21.444417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f427b-8f9a-4d85-88ca-2ec5f6abc138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:21.444752 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f427b-8f9a-4d85-88ca-2ec5f6abc138 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:21.445577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f427b-8f9a-4d85-88ca-2ec5f6abc138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:21.446612 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:46:21.448517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f427b-8f9a-4d85-88ca-2ec5f6abc138 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:21.460659 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64556bef-8f87-4d07-bf38-5c4a8acd4691 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:21.468703 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1046/2090] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:46:21.469861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f427b-8f9a-4d85-88ca-2ec5f6abc138 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:21.470284 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ed053-f218-4aad-a369-25fcf30e5e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:21.471336 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ed053-f218-4aad-a369-25fcf30e5e01 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:21.471754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ed053-f218-4aad-a369-25fcf30e5e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:21.472286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ed053-f218-4aad-a369-25fcf30e5e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:21.473451 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ed053-f218-4aad-a369-25fcf30e5e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:21.476579 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ed053-f218-4aad-a369-25fcf30e5e01 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:21.481479 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f427b-8f9a-4d85-88ca-2ec5f6abc138 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:21.487496 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912397d0-4ec7-4c27-a67e-138957991717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:21.488242 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912397d0-4ec7-4c27-a67e-138957991717 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:21.488543 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912397d0-4ec7-4c27-a67e-138957991717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:21.488804 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912397d0-4ec7-4c27-a67e-138957991717 admin admin] PATH_INFO: `/v3/projects/11b0254ba7d0421e9d9214a1c400466a/users/f72e1a846e864aa8b00edd8ae1a948ff/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:21.500475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1045/2091] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:46:21.511160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ed053-f218-4aad-a369-25fcf30e5e01 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:21.524112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ed053-f218-4aad-a369-25fcf30e5e01 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:21.526795 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912397d0-4ec7-4c27-a67e-138957991717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11b0254ba7d0421e9d9214a1c400466a, user_id=f72e1a846e864aa8b00edd8ae1a948ff, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:21.530399 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912397d0-4ec7-4c27-a67e-138957991717 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:21.532541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504de0fe-5beb-4c7a-a872-96cac410be82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:21.533534 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504de0fe-5beb-4c7a-a872-96cac410be82 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:21.533989 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504de0fe-5beb-4c7a-a872-96cac410be82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:21.534442 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504de0fe-5beb-4c7a-a872-96cac410be82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:21.535592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504de0fe-5beb-4c7a-a872-96cac410be82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:21.538686 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504de0fe-5beb-4c7a-a872-96cac410be82 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:21.549257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1047/2092] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:46:21 2022] PUT /identity/v3/projects/11b0254ba7d0421e9d9214a1c400466a/users/f72e1a846e864aa8b00edd8ae1a948ff/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:46:21.554795 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c27a7-eab6-4d88-8d1e-a44ad56bed41 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:21.555188 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c27a7-eab6-4d88-8d1e-a44ad56bed41 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:21.555559 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c27a7-eab6-4d88-8d1e-a44ad56bed41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:21.560041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1046/2093] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:46:21.566331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504de0fe-5beb-4c7a-a872-96cac410be82 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:21.580695 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-146c27a7-eab6-4d88-8d1e-a44ad56bed41 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:46:21.582708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-146c27a7-eab6-4d88-8d1e-a44ad56bed41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:21.585596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504de0fe-5beb-4c7a-a872-96cac410be82 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:21.683428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16c10a-53c5-43a4-8031-f0502d85c34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:21.688374 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1048/2094] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:46:21.690620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16c10a-53c5-43a4-8031-f0502d85c34a admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:21.691138 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16c10a-53c5-43a4-8031-f0502d85c34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:21.691603 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16c10a-53c5-43a4-8031-f0502d85c34a admin admin] PATH_INFO: `/v3/projects/1c11e40cb7554d419d54161d941a4760/users/6e9a297e7be746e2bff3615033ca286e/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:21.703196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c27a7-eab6-4d88-8d1e-a44ad56bed41 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:21.705907 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-146c27a7-eab6-4d88-8d1e-a44ad56bed41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:21.724772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16c10a-53c5-43a4-8031-f0502d85c34a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c11e40cb7554d419d54161d941a4760, user_id=6e9a297e7be746e2bff3615033ca286e, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:21.732598 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16c10a-53c5-43a4-8031-f0502d85c34a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:21.771832 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1047/2095] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:46:21 2022] PUT /identity/v3/projects/1c11e40cb7554d419d54161d941a4760/users/6e9a297e7be746e2bff3615033ca286e/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:46:21.793583 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ec474-a1fc-4f63-ba1f-6af1be1f6297 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:21.794653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ec474-a1fc-4f63-ba1f-6af1be1f6297 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:21.795173 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ec474-a1fc-4f63-ba1f-6af1be1f6297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:21.851850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-b60ec474-a1fc-4f63-ba1f-6af1be1f6297 None None] MFA Rules not processed for user `6e9a297e7be746e2bff3615033ca286e`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:46:21.853674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c27a7-eab6-4d88-8d1e-a44ad56bed41 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:21.854538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b60ec474-a1fc-4f63-ba1f-6af1be1f6297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:21.951428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd0808-b855-43ce-9268-d362d1a60bda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:21.953521 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd0808-b855-43ce-9268-d362d1a60bda service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:21.954007 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd0808-b855-43ce-9268-d362d1a60bda service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:21.954559 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd0808-b855-43ce-9268-d362d1a60bda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:21.995979 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:46:22.018231 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1048/2096] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:46:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:46:22.032411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ec474-a1fc-4f63-ba1f-6af1be1f6297 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:22.034245 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b60ec474-a1fc-4f63-ba1f-6af1be1f6297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:22.087031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd0808-b855-43ce-9268-d362d1a60bda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:22.088576 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd0808-b855-43ce-9268-d362d1a60bda service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:22.102335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1049/2097] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:46:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:46:22.138248 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ec474-a1fc-4f63-ba1f-6af1be1f6297 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:22.144514 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ef01d-9fcd-4a40-99f7-9507aef42267 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69de9368993f402cb6325e5022e064d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:22.145185 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ef01d-9fcd-4a40-99f7-9507aef42267 service glance] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:22.145461 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ef01d-9fcd-4a40-99f7-9507aef42267 service glance] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:22.145729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ef01d-9fcd-4a40-99f7-9507aef42267 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:22.153189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:46:22.157144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ef01d-9fcd-4a40-99f7-9507aef42267 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:22.159693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ef01d-9fcd-4a40-99f7-9507aef42267 service glance] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:22.183466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1049/2098] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:46:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:46:28.465939 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501b9e0-6749-4f1d-913e-bb996d9e9a3b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:28.466428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501b9e0-6749-4f1d-913e-bb996d9e9a3b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:28.466428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501b9e0-6749-4f1d-913e-bb996d9e9a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:28.500112 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1501b9e0-6749-4f1d-913e-bb996d9e9a3b None None] MFA Rules not processed for user `f72e1a846e864aa8b00edd8ae1a948ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:46:28.501657 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1501b9e0-6749-4f1d-913e-bb996d9e9a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:28.593941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1050/2099] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:46:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:46:28.608783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ef01d-9fcd-4a40-99f7-9507aef42267 service glance] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:28.610228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-789ef01d-9fcd-4a40-99f7-9507aef42267 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:28.742336 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ef01d-9fcd-4a40-99f7-9507aef42267 service glance] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:28.752754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de305455-b4cd-499e-ac60-66fc91ac79b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:28.753965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de305455-b4cd-499e-ac60-66fc91ac79b7 service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:28.754255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de305455-b4cd-499e-ac60-66fc91ac79b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:28.754723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de305455-b4cd-499e-ac60-66fc91ac79b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:28.770192 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de305455-b4cd-499e-ac60-66fc91ac79b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:28.772448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de305455-b4cd-499e-ac60-66fc91ac79b7 service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:28.788202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1050/2100] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:46:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:46:30.682262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b666297-6d23-4688-a359-8eb312a865cf None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:30.682714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b666297-6d23-4688-a359-8eb312a865cf None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:30.682714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b666297-6d23-4688-a359-8eb312a865cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:30.709578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7b666297-6d23-4688-a359-8eb312a865cf None None] MFA Rules not processed for user `f72e1a846e864aa8b00edd8ae1a948ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:46:30.710821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7b666297-6d23-4688-a359-8eb312a865cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:30.718196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1051/2101] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:46:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:46:38.779854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=36 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:46:40.697196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=56 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:00.970988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdf724-7ddb-4c6d-a165-5101aad18c1d None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:00.972534 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdf724-7ddb-4c6d-a165-5101aad18c1d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:00.972929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdf724-7ddb-4c6d-a165-5101aad18c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:01.001001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-26fdf724-7ddb-4c6d-a165-5101aad18c1d None None] MFA Rules not processed for user `8fc940594a1848aa9644b645049a1bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:01.002590 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-26fdf724-7ddb-4c6d-a165-5101aad18c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:01.123130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1051/2102] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Aug 30 14:47:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:01.213342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b666297-6d23-4688-a359-8eb312a865cf None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:01.217333 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7b666297-6d23-4688-a359-8eb312a865cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:01.324630 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b666297-6d23-4688-a359-8eb312a865cf None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:01.327234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68c7a2-ea20-415f-8a28-0708bb955a36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d98f5adfaf410580481417e7ba1638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:01.327872 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68c7a2-ea20-415f-8a28-0708bb955a36 service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:01.328264 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68c7a2-ea20-415f-8a28-0708bb955a36 service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:01.328860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68c7a2-ea20-415f-8a28-0708bb955a36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:01.339257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68c7a2-ea20-415f-8a28-0708bb955a36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:01.341000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68c7a2-ea20-415f-8a28-0708bb955a36 service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:01.351651 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1052/2103] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Aug 30 14:47:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:47:02.053138 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3ef33-f75d-4afa-b8da-573ab5f23cef None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:02.053651 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3ef33-f75d-4afa-b8da-573ab5f23cef None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:02.053759 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3ef33-f75d-4afa-b8da-573ab5f23cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:02.082995 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-27f3ef33-f75d-4afa-b8da-573ab5f23cef None None] MFA Rules not processed for user `8fc940594a1848aa9644b645049a1bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:02.084169 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-27f3ef33-f75d-4afa-b8da-573ab5f23cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:02.091479 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1052/2104] 10.222.0.25 () {58 vars in 967 bytes} [Tue Aug 30 14:47:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:02.105395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68c7a2-ea20-415f-8a28-0708bb955a36 service swift] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:02.116117 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68c7a2-ea20-415f-8a28-0708bb955a36 service swift] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:02.121178 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0de9d-7b18-4472-b57a-58be7b7febac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d98f5adfaf410580481417e7ba1638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:02.121785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0de9d-7b18-4472-b57a-58be7b7febac service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:02.122090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0de9d-7b18-4472-b57a-58be7b7febac service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:02.122351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0de9d-7b18-4472-b57a-58be7b7febac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:02.130869 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0de9d-7b18-4472-b57a-58be7b7febac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:02.132570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0de9d-7b18-4472-b57a-58be7b7febac service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:02.141252 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1053/2105] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Aug 30 14:47:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:47:02.196961 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3ef33-f75d-4afa-b8da-573ab5f23cef None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:02.198507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-27f3ef33-f75d-4afa-b8da-573ab5f23cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:02.214654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3ef33-f75d-4afa-b8da-573ab5f23cef None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:02.220940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514fd86-bc82-4505-90a4-c5a3ecadd846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:02.221689 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514fd86-bc82-4505-90a4-c5a3ecadd846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:02.222331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514fd86-bc82-4505-90a4-c5a3ecadd846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:02.223466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514fd86-bc82-4505-90a4-c5a3ecadd846 admin admin] PATH_INFO: `/v3/users/075e43152aaf44d0b47d2d397fcda602` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:02.225576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4514fd86-bc82-4505-90a4-c5a3ecadd846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=075e43152aaf44d0b47d2d397fcda602)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:02.228145 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4514fd86-bc82-4505-90a4-c5a3ecadd846 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:02.299242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4514fd86-bc82-4505-90a4-c5a3ecadd846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '075e43152aaf44d0b47d2d397fcda602'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:02.299924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4514fd86-bc82-4505-90a4-c5a3ecadd846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '075e43152aaf44d0b47d2d397fcda602'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:02.300727 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4514fd86-bc82-4505-90a4-c5a3ecadd846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '075e43152aaf44d0b47d2d397fcda602'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:02.310093 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4514fd86-bc82-4505-90a4-c5a3ecadd846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '075e43152aaf44d0b47d2d397fcda602'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:02.322838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4514fd86-bc82-4505-90a4-c5a3ecadd846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '075e43152aaf44d0b47d2d397fcda602'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:02.337936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1053/2106] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:47:02 2022] DELETE /identity/v3/users/075e43152aaf44d0b47d2d397fcda602 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:47:02.342764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee17f18-3617-4db0-b091-cf8ce1af847a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:02.343258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee17f18-3617-4db0-b091-cf8ce1af847a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:02.343352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee17f18-3617-4db0-b091-cf8ce1af847a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:02.369086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-fee17f18-3617-4db0-b091-cf8ce1af847a None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:02.371104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-fee17f18-3617-4db0-b091-cf8ce1af847a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:02.445781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25ff89-e657-4c7e-ba9b-56cd9417a0e9 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:02.446564 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25ff89-e657-4c7e-ba9b-56cd9417a0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:02.447373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25ff89-e657-4c7e-ba9b-56cd9417a0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:02.455529 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1054/2107] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:02.467439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee17f18-3617-4db0-b091-cf8ce1af847a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:02.470118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-fee17f18-3617-4db0-b091-cf8ce1af847a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:02.478143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-6e25ff89-e657-4c7e-ba9b-56cd9417a0e9 None None] MFA Rules not processed for user `8fc940594a1848aa9644b645049a1bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:02.479691 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6e25ff89-e657-4c7e-ba9b-56cd9417a0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:02.565449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee17f18-3617-4db0-b091-cf8ce1af847a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:02.568655 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1054/2108] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Aug 30 14:47:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:02.570312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957c061-24d5-48e4-b3c8-48d724cdfd8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:02.570948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957c061-24d5-48e4-b3c8-48d724cdfd8c service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:02.571223 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957c061-24d5-48e4-b3c8-48d724cdfd8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:02.571491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957c061-24d5-48e4-b3c8-48d724cdfd8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:02.586082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957c061-24d5-48e4-b3c8-48d724cdfd8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:02.589135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957c061-24d5-48e4-b3c8-48d724cdfd8c service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:02.592475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25ff89-e657-4c7e-ba9b-56cd9417a0e9 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:02.594587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6e25ff89-e657-4c7e-ba9b-56cd9417a0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:02.607070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1055/2109] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:02.618950 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:02.692316 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25ff89-e657-4c7e-ba9b-56cd9417a0e9 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:02.698324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9586d00-cd7b-4136-9847-5f49d30bc02c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d98f5adfaf410580481417e7ba1638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:02.698919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9586d00-cd7b-4136-9847-5f49d30bc02c service swift] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:02.699197 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9586d00-cd7b-4136-9847-5f49d30bc02c service swift] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:02.699479 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9586d00-cd7b-4136-9847-5f49d30bc02c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:02.713429 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9586d00-cd7b-4136-9847-5f49d30bc02c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:02.715682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9586d00-cd7b-4136-9847-5f49d30bc02c service swift] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:02.726924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1055/2110] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Aug 30 14:47:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 14:47:03.003158 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957c061-24d5-48e4-b3c8-48d724cdfd8c service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:03.006071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6957c061-24d5-48e4-b3c8-48d724cdfd8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:03.058457 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957c061-24d5-48e4-b3c8-48d724cdfd8c service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:03.067896 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93c712-1c91-4d45-94c9-97ae62016b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:03.069666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93c712-1c91-4d45-94c9-97ae62016b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:03.070148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93c712-1c91-4d45-94c9-97ae62016b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:03.070611 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93c712-1c91-4d45-94c9-97ae62016b61 admin admin] PATH_INFO: `/v3/projects/13cd13ed995243b4ae4a97c10ddeff01` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:03.072775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93c712-1c91-4d45-94c9-97ae62016b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13cd13ed995243b4ae4a97c10ddeff01)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:03.076653 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93c712-1c91-4d45-94c9-97ae62016b61 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:03.118927 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:03.182220 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9b93c712-1c91-4d45-94c9-97ae62016b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13cd13ed995243b4ae4a97c10ddeff01', 'request_id': 'req-9b93c712-1c91-4d45-94c9-97ae62016b61'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:03.233332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9b93c712-1c91-4d45-94c9-97ae62016b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13cd13ed995243b4ae4a97c10ddeff01', 'request_id': 'req-9b93c712-1c91-4d45-94c9-97ae62016b61'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:03.255060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9b93c712-1c91-4d45-94c9-97ae62016b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13cd13ed995243b4ae4a97c10ddeff01', 'request_id': 'req-9b93c712-1c91-4d45-94c9-97ae62016b61'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:03.258316 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a75705-cdfc-4136-b612-0d63bb385315 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:03.259105 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a75705-cdfc-4136-b612-0d63bb385315 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:03.260813 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a75705-cdfc-4136-b612-0d63bb385315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:03.279926 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9b93c712-1c91-4d45-94c9-97ae62016b61 admin admin] The token cache is being invalidate because project 13cd13ed995243b4ae4a97c10ddeff01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:47:03.280408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9b93c712-1c91-4d45-94c9-97ae62016b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:03.281223 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9b93c712-1c91-4d45-94c9-97ae62016b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:03.284600 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1056/2111] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:47:03 2022] DELETE /identity/v3/projects/13cd13ed995243b4ae4a97c10ddeff01 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:03.292639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cca93-56fe-46e4-abe9-5bd2a1b855f8 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:03.293212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cca93-56fe-46e4-abe9-5bd2a1b855f8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:03.293662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cca93-56fe-46e4-abe9-5bd2a1b855f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:03.352143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-15a75705-cdfc-4136-b612-0d63bb385315 None None] MFA Rules not processed for user `8fc940594a1848aa9644b645049a1bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:03.354375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a95cca93-56fe-46e4-abe9-5bd2a1b855f8 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:03.356234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a95cca93-56fe-46e4-abe9-5bd2a1b855f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:03.356445 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-15a75705-cdfc-4136-b612-0d63bb385315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:03.510582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1056/2112] 10.222.0.25 () {58 vars in 967 bytes} [Tue Aug 30 14:47:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:03.537643 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a75705-cdfc-4136-b612-0d63bb385315 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:03.540494 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-15a75705-cdfc-4136-b612-0d63bb385315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:03.565940 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1057/2113] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:03.577265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d58e00-b308-4b5d-aa61-540d51b169d7 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:03.577955 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d58e00-b308-4b5d-aa61-540d51b169d7 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:03.578654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d58e00-b308-4b5d-aa61-540d51b169d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:03.614882 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-08d58e00-b308-4b5d-aa61-540d51b169d7 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:03.616994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-08d58e00-b308-4b5d-aa61-540d51b169d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:03.651879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1058/2114] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:47:03.656998 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d58e00-b308-4b5d-aa61-540d51b169d7 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:03.694697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d58e00-b308-4b5d-aa61-540d51b169d7 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:03.704605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe562a1-fbf8-433a-9080-b5ae82fb394e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:03.706110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe562a1-fbf8-433a-9080-b5ae82fb394e admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:03.707117 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe562a1-fbf8-433a-9080-b5ae82fb394e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:03.707566 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe562a1-fbf8-433a-9080-b5ae82fb394e admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:03.709625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe562a1-fbf8-433a-9080-b5ae82fb394e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:47:03.710142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe562a1-fbf8-433a-9080-b5ae82fb394e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:03.715255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe562a1-fbf8-433a-9080-b5ae82fb394e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:03.764147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1059/2115] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:47:03.766022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe562a1-fbf8-433a-9080-b5ae82fb394e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:03.784995 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a75705-cdfc-4136-b612-0d63bb385315 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:03.788594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe562a1-fbf8-433a-9080-b5ae82fb394e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:03.805777 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0040c-9c9f-4ed7-9fa7-3a612c537a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:03.806425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0040c-9c9f-4ed7-9fa7-3a612c537a71 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:03.806717 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0040c-9c9f-4ed7-9fa7-3a612c537a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:03.806977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0040c-9c9f-4ed7-9fa7-3a612c537a71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:03.807495 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9083414-7132-4dd5-96c0-d3dbcbdcbb85 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d98f5adfaf410580481417e7ba1638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:03.808821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9083414-7132-4dd5-96c0-d3dbcbdcbb85 service swift] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:03.809268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9083414-7132-4dd5-96c0-d3dbcbdcbb85 service swift] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:03.810148 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9083414-7132-4dd5-96c0-d3dbcbdcbb85 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:03.812390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0040c-9c9f-4ed7-9fa7-3a612c537a71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:03.816062 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0040c-9c9f-4ed7-9fa7-3a612c537a71 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:03.834649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9083414-7132-4dd5-96c0-d3dbcbdcbb85 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:03.839843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9083414-7132-4dd5-96c0-d3dbcbdcbb85 service swift] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:03.868069 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1057/2116] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Aug 30 14:47:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:47:03.887179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1060/2117] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:47:03 2022] POST /identity/v3/projects => generated 389 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:47:03.892514 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9083414-7132-4dd5-96c0-d3dbcbdcbb85 service swift] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:03.905223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9083414-7132-4dd5-96c0-d3dbcbdcbb85 service swift] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:04.002447 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:04.038536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09b70e-f00f-4c13-a1a1-bfb18ff44455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:04.039730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09b70e-f00f-4c13-a1a1-bfb18ff44455 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:04.040300 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09b70e-f00f-4c13-a1a1-bfb18ff44455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:04.040789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09b70e-f00f-4c13-a1a1-bfb18ff44455 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:04.042101 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09b70e-f00f-4c13-a1a1-bfb18ff44455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:04.045767 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09b70e-f00f-4c13-a1a1-bfb18ff44455 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:04.088730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1058/2118] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:47:03 2022] POST /identity/v3/users => generated 295 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:47:04.089657 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0040c-9c9f-4ed7-9fa7-3a612c537a71 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:04.103950 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0040c-9c9f-4ed7-9fa7-3a612c537a71 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:04.111124 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bc392-fa18-4fe3-b5ba-18d7e925d083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:04.111939 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bc392-fa18-4fe3-b5ba-18d7e925d083 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:04.112516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bc392-fa18-4fe3-b5ba-18d7e925d083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:04.113230 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bc392-fa18-4fe3-b5ba-18d7e925d083 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:04.114405 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bc392-fa18-4fe3-b5ba-18d7e925d083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:04.116408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bc392-fa18-4fe3-b5ba-18d7e925d083 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:04.133220 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1061/2119] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:47:04.137993 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09b70e-f00f-4c13-a1a1-bfb18ff44455 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:04.160189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09b70e-f00f-4c13-a1a1-bfb18ff44455 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:04.168480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f726540-0de2-4423-9802-42db750634cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:04.169488 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f726540-0de2-4423-9802-42db750634cb admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:04.169913 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f726540-0de2-4423-9802-42db750634cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:04.170396 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f726540-0de2-4423-9802-42db750634cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:04.171357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f726540-0de2-4423-9802-42db750634cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:04.174228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f726540-0de2-4423-9802-42db750634cb admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:04.194204 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1059/2120] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:47:04.198776 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bc392-fa18-4fe3-b5ba-18d7e925d083 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:04.214073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bc392-fa18-4fe3-b5ba-18d7e925d083 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:04.221965 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf1948-095c-43d9-9dcc-47a79504332b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:04.223135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf1948-095c-43d9-9dcc-47a79504332b admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:04.223539 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf1948-095c-43d9-9dcc-47a79504332b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:04.223958 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf1948-095c-43d9-9dcc-47a79504332b admin admin] PATH_INFO: `/v3/projects/084892e1b0834abba4f84624cae77379/users/23fe3372d1cd4d2380b22fedd5636612/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:04.261958 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf1948-095c-43d9-9dcc-47a79504332b admin admin] RBAC: Authorizing `identity:create_grant(project_id=084892e1b0834abba4f84624cae77379, user_id=23fe3372d1cd4d2380b22fedd5636612, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:04.267867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf1948-095c-43d9-9dcc-47a79504332b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:04.294999 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1062/2121] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:47:04 2022] PUT /identity/v3/projects/084892e1b0834abba4f84624cae77379/users/23fe3372d1cd4d2380b22fedd5636612/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:04.311859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e17c51-8c50-4fdf-8b1f-4261d4abc52f None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:04.311859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e17c51-8c50-4fdf-8b1f-4261d4abc52f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:04.311859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e17c51-8c50-4fdf-8b1f-4261d4abc52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:04.384907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-d4e17c51-8c50-4fdf-8b1f-4261d4abc52f None None] MFA Rules not processed for user `23fe3372d1cd4d2380b22fedd5636612`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:04.386456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d4e17c51-8c50-4fdf-8b1f-4261d4abc52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:04.532272 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1060/2122] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:04.561677 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf1948-095c-43d9-9dcc-47a79504332b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:04.568612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-52cf1948-095c-43d9-9dcc-47a79504332b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:04.623520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:04.681972 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf1948-095c-43d9-9dcc-47a79504332b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:04.697870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61254d9d-dbb4-44da-8c40-e28e25c0a4fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:04.699083 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61254d9d-dbb4-44da-8c40-e28e25c0a4fb service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:04.699465 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61254d9d-dbb4-44da-8c40-e28e25c0a4fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:04.699844 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61254d9d-dbb4-44da-8c40-e28e25c0a4fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:04.714408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61254d9d-dbb4-44da-8c40-e28e25c0a4fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:04.716142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61254d9d-dbb4-44da-8c40-e28e25c0a4fb service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:04.728692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1063/2123] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:04 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:14.511926 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=48,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:14.723851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=14 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:15.450803 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d56528-9daf-4bf5-aa6e-8667a1b74afb None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:15.451294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d56528-9daf-4bf5-aa6e-8667a1b74afb None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:15.451544 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d56528-9daf-4bf5-aa6e-8667a1b74afb None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:15.453289 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1061/2124] 10.222.0.25 () {54 vars in 961 bytes} [Tue Aug 30 14:47:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:47:15.460247 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57ca2-57c1-4cdf-ae68-783697c2111d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:15.460504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57ca2-57c1-4cdf-ae68-783697c2111d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:15.461018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57ca2-57c1-4cdf-ae68-783697c2111d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:15.484639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-88c57ca2-57c1-4cdf-ae68-783697c2111d None None] MFA Rules not processed for user `d7cdd6b1ee9f4e97beb9bb037e446c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:15.485879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-88c57ca2-57c1-4cdf-ae68-783697c2111d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:15.609161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1064/2125] 10.222.0.25 () {58 vars in 1058 bytes} [Tue Aug 30 14:47:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:15.610560 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d56528-9daf-4bf5-aa6e-8667a1b74afb None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:15.629811 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d56528-9daf-4bf5-aa6e-8667a1b74afb None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:15.635573 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff57d5-de10-4261-8521-43ddcddefafb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cdd6b1ee9f4e97beb9bb037e446c8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:15.636255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff57d5-de10-4261-8521-43ddcddefafb service placement] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:15.636533 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff57d5-de10-4261-8521-43ddcddefafb service placement] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:15.636805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff57d5-de10-4261-8521-43ddcddefafb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:15.638751 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-73ff57d5-de10-4261-8521-43ddcddefafb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:15.653902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff57d5-de10-4261-8521-43ddcddefafb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:15.656679 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff57d5-de10-4261-8521-43ddcddefafb service placement] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:15.674090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1062/2126] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:16.676900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c57ca2-57c1-4cdf-ae68-783697c2111d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:16.679171 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-88c57ca2-57c1-4cdf-ae68-783697c2111d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:16.777582 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c57ca2-57c1-4cdf-ae68-783697c2111d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:16.782611 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3405f-123f-441b-8159-3350533af9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:16.783360 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3405f-123f-441b-8159-3350533af9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:16.783751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3405f-123f-441b-8159-3350533af9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:16.784172 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3405f-123f-441b-8159-3350533af9a4 admin admin] PATH_INFO: `/v3/users/5e260fff0fab4cd0979047116a1e3e32` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:16.785735 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3405f-123f-441b-8159-3350533af9a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e260fff0fab4cd0979047116a1e3e32)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:16.788774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3405f-123f-441b-8159-3350533af9a4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:16.846697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5ae3405f-123f-441b-8159-3350533af9a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e260fff0fab4cd0979047116a1e3e32'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:16.854733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5ae3405f-123f-441b-8159-3350533af9a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e260fff0fab4cd0979047116a1e3e32'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:16.863271 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5ae3405f-123f-441b-8159-3350533af9a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e260fff0fab4cd0979047116a1e3e32'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:16.863954 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5ae3405f-123f-441b-8159-3350533af9a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e260fff0fab4cd0979047116a1e3e32'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:16.879865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5ae3405f-123f-441b-8159-3350533af9a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e260fff0fab4cd0979047116a1e3e32'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:16.890097 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1065/2127] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:47:16 2022] DELETE /identity/v3/users/5e260fff0fab4cd0979047116a1e3e32 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:47:16.896285 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7dad12-d813-4b8c-aecb-7def983ede21 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:16.896834 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7dad12-d813-4b8c-aecb-7def983ede21 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:16.896834 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7dad12-d813-4b8c-aecb-7def983ede21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:16.921256 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-6f7dad12-d813-4b8c-aecb-7def983ede21 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:16.922606 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6f7dad12-d813-4b8c-aecb-7def983ede21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:17.002924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1063/2128] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:17.017229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3405f-123f-441b-8159-3350533af9a4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:17.019532 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5ae3405f-123f-441b-8159-3350533af9a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:17.040596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:17.104051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3405f-123f-441b-8159-3350533af9a4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:17.108608 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ff809-667d-4613-a3f3-338d979c427b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:17.109271 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ff809-667d-4613-a3f3-338d979c427b service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:17.109550 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ff809-667d-4613-a3f3-338d979c427b service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:17.109821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ff809-667d-4613-a3f3-338d979c427b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:17.123075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ff809-667d-4613-a3f3-338d979c427b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:17.124694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ff809-667d-4613-a3f3-338d979c427b service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:17.146524 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1066/2129] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:47:17.376417 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7dad12-d813-4b8c-aecb-7def983ede21 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:17.378123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6f7dad12-d813-4b8c-aecb-7def983ede21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:17.393234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7dad12-d813-4b8c-aecb-7def983ede21 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:17.399653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf4c59-5992-42d3-8d53-8add5615a626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:17.400433 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf4c59-5992-42d3-8d53-8add5615a626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:17.400832 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf4c59-5992-42d3-8d53-8add5615a626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:17.400832 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf4c59-5992-42d3-8d53-8add5615a626 admin admin] PATH_INFO: `/v3/projects/39e7bfd5fca640109a055032971dfcb7` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:17.402258 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf4c59-5992-42d3-8d53-8add5615a626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39e7bfd5fca640109a055032971dfcb7)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:17.404386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf4c59-5992-42d3-8d53-8add5615a626 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:17.467651 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cabf4c59-5992-42d3-8d53-8add5615a626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39e7bfd5fca640109a055032971dfcb7', 'request_id': 'req-cabf4c59-5992-42d3-8d53-8add5615a626'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:17.474513 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cabf4c59-5992-42d3-8d53-8add5615a626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39e7bfd5fca640109a055032971dfcb7', 'request_id': 'req-cabf4c59-5992-42d3-8d53-8add5615a626'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:17.485061 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cabf4c59-5992-42d3-8d53-8add5615a626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39e7bfd5fca640109a055032971dfcb7', 'request_id': 'req-cabf4c59-5992-42d3-8d53-8add5615a626'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:17.504302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cabf4c59-5992-42d3-8d53-8add5615a626 admin admin] The token cache is being invalidate because project 39e7bfd5fca640109a055032971dfcb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:47:17.504735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cabf4c59-5992-42d3-8d53-8add5615a626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:17.505572 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cabf4c59-5992-42d3-8d53-8add5615a626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:17.507722 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1064/2130] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:47:17 2022] DELETE /identity/v3/projects/39e7bfd5fca640109a055032971dfcb7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:17.514099 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86c575-5f61-474d-a515-39978e31b4c4 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:17.514397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86c575-5f61-474d-a515-39978e31b4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:17.514538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86c575-5f61-474d-a515-39978e31b4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:17.537849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-5b86c575-5f61-474d-a515-39978e31b4c4 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:17.539213 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5b86c575-5f61-474d-a515-39978e31b4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:17.620112 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1067/2131] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:17.630432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490854c-d545-494f-bfae-29de5474bbdd None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:17.630910 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490854c-d545-494f-bfae-29de5474bbdd None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:17.631382 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490854c-d545-494f-bfae-29de5474bbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:17.666359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-3490854c-d545-494f-bfae-29de5474bbdd None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:17.668568 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-3490854c-d545-494f-bfae-29de5474bbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:17.679534 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1065/2132] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:47:17.685530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86c575-5f61-474d-a515-39978e31b4c4 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:17.701457 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86c575-5f61-474d-a515-39978e31b4c4 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:17.706710 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa1a3c-aa0f-433c-bee5-1bdc8a792b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:17.707506 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa1a3c-aa0f-433c-bee5-1bdc8a792b24 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:17.707837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa1a3c-aa0f-433c-bee5-1bdc8a792b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:17.708161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa1a3c-aa0f-433c-bee5-1bdc8a792b24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:17.708833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa1a3c-aa0f-433c-bee5-1bdc8a792b24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:47:17.709316 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa1a3c-aa0f-433c-bee5-1bdc8a792b24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:17.711386 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa1a3c-aa0f-433c-bee5-1bdc8a792b24 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:17.725474 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1068/2133] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:47:17.733090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490854c-d545-494f-bfae-29de5474bbdd None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:17.751747 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490854c-d545-494f-bfae-29de5474bbdd None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:17.753582 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa1a3c-aa0f-433c-bee5-1bdc8a792b24 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:17.755357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-78fa1a3c-aa0f-433c-bee5-1bdc8a792b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:17.759654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367ae48-f9b7-4281-bb09-263f5520a3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:17.760851 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367ae48-f9b7-4281-bb09-263f5520a3a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:17.761366 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367ae48-f9b7-4281-bb09-263f5520a3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:17.761919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367ae48-f9b7-4281-bb09-263f5520a3a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:17.763149 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f367ae48-f9b7-4281-bb09-263f5520a3a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:17.766965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f367ae48-f9b7-4281-bb09-263f5520a3a6 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:17.769957 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa1a3c-aa0f-433c-bee5-1bdc8a792b24 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:17.775680 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46cbe8a-5de3-470d-8ae8-4bfe52770a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:17.776387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cbe8a-5de3-470d-8ae8-4bfe52770a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:17.776655 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cbe8a-5de3-470d-8ae8-4bfe52770a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:17.776924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cbe8a-5de3-470d-8ae8-4bfe52770a5e admin admin] PATH_INFO: `/v3/users/6b76f320445b4723908a44915f879add` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:17.778253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46cbe8a-5de3-470d-8ae8-4bfe52770a5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b76f320445b4723908a44915f879add)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:17.780268 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46cbe8a-5de3-470d-8ae8-4bfe52770a5e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:17.810471 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1066/2134] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:47:17 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:47:17.815775 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367ae48-f9b7-4281-bb09-263f5520a3a6 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:17.843116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c46cbe8a-5de3-470d-8ae8-4bfe52770a5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b76f320445b4723908a44915f879add'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:17.854451 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c46cbe8a-5de3-470d-8ae8-4bfe52770a5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b76f320445b4723908a44915f879add'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:17.866134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c46cbe8a-5de3-470d-8ae8-4bfe52770a5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b76f320445b4723908a44915f879add'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:17.869170 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c46cbe8a-5de3-470d-8ae8-4bfe52770a5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b76f320445b4723908a44915f879add'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:17.884576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:17.892781 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c46cbe8a-5de3-470d-8ae8-4bfe52770a5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b76f320445b4723908a44915f879add'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:17.906340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1069/2135] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:47:17 2022] DELETE /identity/v3/users/6b76f320445b4723908a44915f879add => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:47:17.909506 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367ae48-f9b7-4281-bb09-263f5520a3a6 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:17.912013 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5c6af-b0a6-4047-b9b6-1a0a9d008c27 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:17.912471 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5c6af-b0a6-4047-b9b6-1a0a9d008c27 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:17.912749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5c6af-b0a6-4047-b9b6-1a0a9d008c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:17.936420 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-aba5c6af-b0a6-4047-b9b6-1a0a9d008c27 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:17.937695 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-aba5c6af-b0a6-4047-b9b6-1a0a9d008c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:17.952837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:17.995451 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1786e7-4244-490b-a95a-0ab9d62532a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:17.996124 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1786e7-4244-490b-a95a-0ab9d62532a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:17.996399 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1786e7-4244-490b-a95a-0ab9d62532a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:17.996692 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1786e7-4244-490b-a95a-0ab9d62532a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:17.997370 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1786e7-4244-490b-a95a-0ab9d62532a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:17.999386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1786e7-4244-490b-a95a-0ab9d62532a2 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:18.034831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1070/2136] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:47:18.035109 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1067/2137] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:47:17 2022] POST /identity/v3/users => generated 292 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:47:18.038672 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5c6af-b0a6-4047-b9b6-1a0a9d008c27 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:18.040364 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-aba5c6af-b0a6-4047-b9b6-1a0a9d008c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:18.042103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1786e7-4244-490b-a95a-0ab9d62532a2 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:18.044459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-3f1786e7-4244-490b-a95a-0ab9d62532a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:18.053285 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5c6af-b0a6-4047-b9b6-1a0a9d008c27 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:18.058598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e558b-afec-49f2-920e-1d8437d9cf6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:18.059633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e558b-afec-49f2-920e-1d8437d9cf6c admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:18.059955 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e558b-afec-49f2-920e-1d8437d9cf6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:18.060324 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e558b-afec-49f2-920e-1d8437d9cf6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:18.060889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e558b-afec-49f2-920e-1d8437d9cf6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:18.062569 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e558b-afec-49f2-920e-1d8437d9cf6c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:18.074195 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1071/2138] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:47:18.078498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e558b-afec-49f2-920e-1d8437d9cf6c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:18.089574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e558b-afec-49f2-920e-1d8437d9cf6c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:18.095039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a9c02-b32a-47d2-83f8-3fcec2a31055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:18.095676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a9c02-b32a-47d2-83f8-3fcec2a31055 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:18.095955 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a9c02-b32a-47d2-83f8-3fcec2a31055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:18.096265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a9c02-b32a-47d2-83f8-3fcec2a31055 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:18.096920 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a9c02-b32a-47d2-83f8-3fcec2a31055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:18.098589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a9c02-b32a-47d2-83f8-3fcec2a31055 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:18.110051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1072/2139] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:47:18.114541 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a9c02-b32a-47d2-83f8-3fcec2a31055 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:18.124711 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1786e7-4244-490b-a95a-0ab9d62532a2 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:18.125182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a9c02-b32a-47d2-83f8-3fcec2a31055 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:18.128868 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042a7bb-8e59-46bb-8651-7ea1c8c9d150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:18.129510 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042a7bb-8e59-46bb-8651-7ea1c8c9d150 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:18.129787 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042a7bb-8e59-46bb-8651-7ea1c8c9d150 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:18.130060 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042a7bb-8e59-46bb-8651-7ea1c8c9d150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:18.130200 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7d6e2-4712-4da2-9526-b378f6754a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:18.130857 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7d6e2-4712-4da2-9526-b378f6754a7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:18.131143 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7d6e2-4712-4da2-9526-b378f6754a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:18.131416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7d6e2-4712-4da2-9526-b378f6754a7f admin admin] PATH_INFO: `/v3/projects/30338c95ad8d476b89bbec65ad188b2d/users/eb1aa3917d3d427e91f0805e5d6ab4eb/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:18.145389 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042a7bb-8e59-46bb-8651-7ea1c8c9d150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:18.146908 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042a7bb-8e59-46bb-8651-7ea1c8c9d150 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:18.149087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7d6e2-4712-4da2-9526-b378f6754a7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=30338c95ad8d476b89bbec65ad188b2d, user_id=eb1aa3917d3d427e91f0805e5d6ab4eb, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:18.151805 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7d6e2-4712-4da2-9526-b378f6754a7f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:18.161527 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1068/2140] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:18.170861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1073/2141] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:47:18 2022] PUT /identity/v3/projects/30338c95ad8d476b89bbec65ad188b2d/users/eb1aa3917d3d427e91f0805e5d6ab4eb/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:47:18.176785 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec9585-b163-4b72-9f76-a2e16e5055d4 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:18.177208 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec9585-b163-4b72-9f76-a2e16e5055d4 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:18.177600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec9585-b163-4b72-9f76-a2e16e5055d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:18.210302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-9fec9585-b163-4b72-9f76-a2e16e5055d4 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:18.212560 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-9fec9585-b163-4b72-9f76-a2e16e5055d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:18.314670 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1069/2142] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:47:18.329489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7d6e2-4712-4da2-9526-b378f6754a7f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:18.332452 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a7c7d6e2-4712-4da2-9526-b378f6754a7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:18.435385 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7d6e2-4712-4da2-9526-b378f6754a7f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:18.439533 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae92ac-eb08-424f-b545-56dc0e5015a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:18.440211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae92ac-eb08-424f-b545-56dc0e5015a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:18.440485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae92ac-eb08-424f-b545-56dc0e5015a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:18.440752 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae92ac-eb08-424f-b545-56dc0e5015a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:18.452212 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae92ac-eb08-424f-b545-56dc0e5015a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:18.454118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae92ac-eb08-424f-b545-56dc0e5015a6 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:18.471125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1074/2143] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:47:18.563152 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec9585-b163-4b72-9f76-a2e16e5055d4 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:18.565279 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-9fec9585-b163-4b72-9f76-a2e16e5055d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:18.595183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec9585-b163-4b72-9f76-a2e16e5055d4 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:18.600866 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d51bc-f4eb-49c4-9ada-6a1180a16fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:18.602854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d51bc-f4eb-49c4-9ada-6a1180a16fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:18.603175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d51bc-f4eb-49c4-9ada-6a1180a16fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:18.603415 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d51bc-f4eb-49c4-9ada-6a1180a16fb9 admin admin] PATH_INFO: `/v3/projects/fa21c1ec3db54b12b2965e81be297713` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:18.604872 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d51bc-f4eb-49c4-9ada-6a1180a16fb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa21c1ec3db54b12b2965e81be297713)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:18.607035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d51bc-f4eb-49c4-9ada-6a1180a16fb9 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:18.638504 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:18.684780 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-089d51bc-f4eb-49c4-9ada-6a1180a16fb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa21c1ec3db54b12b2965e81be297713', 'request_id': 'req-089d51bc-f4eb-49c4-9ada-6a1180a16fb9'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:18.693132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-089d51bc-f4eb-49c4-9ada-6a1180a16fb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa21c1ec3db54b12b2965e81be297713', 'request_id': 'req-089d51bc-f4eb-49c4-9ada-6a1180a16fb9'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:18.702540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-089d51bc-f4eb-49c4-9ada-6a1180a16fb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa21c1ec3db54b12b2965e81be297713', 'request_id': 'req-089d51bc-f4eb-49c4-9ada-6a1180a16fb9'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:18.725985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-089d51bc-f4eb-49c4-9ada-6a1180a16fb9 admin admin] The token cache is being invalidate because project fa21c1ec3db54b12b2965e81be297713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:47:18.726478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-089d51bc-f4eb-49c4-9ada-6a1180a16fb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:18.727569 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-089d51bc-f4eb-49c4-9ada-6a1180a16fb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:18.730035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1070/2144] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:47:18 2022] DELETE /identity/v3/projects/fa21c1ec3db54b12b2965e81be297713 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:18.739406 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fcae4-c296-480a-8cda-fd7318fd5956 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:18.740190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fcae4-c296-480a-8cda-fd7318fd5956 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:18.740590 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fcae4-c296-480a-8cda-fd7318fd5956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:18.775418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0e6fcae4-c296-480a-8cda-fd7318fd5956 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:18.776743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0e6fcae4-c296-480a-8cda-fd7318fd5956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:18.854750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1075/2145] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:18.863224 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb1695-d8fd-49d8-843a-3e15826891fe None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:18.863505 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb1695-d8fd-49d8-843a-3e15826891fe None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:18.863773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb1695-d8fd-49d8-843a-3e15826891fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:18.890427 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-b6cb1695-d8fd-49d8-843a-3e15826891fe None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:18.891704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b6cb1695-d8fd-49d8-843a-3e15826891fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:18.900724 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1071/2146] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:18.905800 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fcae4-c296-480a-8cda-fd7318fd5956 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:18.918227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fcae4-c296-480a-8cda-fd7318fd5956 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:18.924379 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a44133-8389-4605-bba7-5770057f230a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:18.925284 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a44133-8389-4605-bba7-5770057f230a admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:18.925493 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a44133-8389-4605-bba7-5770057f230a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:18.925717 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a44133-8389-4605-bba7-5770057f230a admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:18.926392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a44133-8389-4605-bba7-5770057f230a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:47:18.926691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a44133-8389-4605-bba7-5770057f230a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:18.928385 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a44133-8389-4605-bba7-5770057f230a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:18.944155 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1076/2147] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:47:18.951337 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb1695-d8fd-49d8-843a-3e15826891fe None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:18.968772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb1695-d8fd-49d8-843a-3e15826891fe None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:18.977326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2755071-bf68-43c1-912a-8873413f2e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:18.978384 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2755071-bf68-43c1-912a-8873413f2e5e admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:18.978858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2755071-bf68-43c1-912a-8873413f2e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:18.979388 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2755071-bf68-43c1-912a-8873413f2e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:18.980698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2755071-bf68-43c1-912a-8873413f2e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:18.984604 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2755071-bf68-43c1-912a-8873413f2e5e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:19.045314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1072/2148] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:47:18 2022] POST /identity/v3/projects => generated 381 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:47:19.046646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a44133-8389-4605-bba7-5770057f230a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:19.052487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:19.062658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a44133-8389-4605-bba7-5770057f230a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:19.165378 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b4b6a-c5ff-4a80-88c3-2f66fff920ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:19.168712 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b4b6a-c5ff-4a80-88c3-2f66fff920ac admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:19.169149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b4b6a-c5ff-4a80-88c3-2f66fff920ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:19.169590 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b4b6a-c5ff-4a80-88c3-2f66fff920ac admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:19.170740 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b4b6a-c5ff-4a80-88c3-2f66fff920ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:19.174216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b4b6a-c5ff-4a80-88c3-2f66fff920ac admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:19.216606 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1077/2149] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:47:19 2022] POST /identity/v3/users => generated 291 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:47:19.218262 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2755071-bf68-43c1-912a-8873413f2e5e admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:19.236109 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2755071-bf68-43c1-912a-8873413f2e5e admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:19.264278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc31a5-17be-4a76-9989-9390d8b7af84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:19.265241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc31a5-17be-4a76-9989-9390d8b7af84 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:19.265681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc31a5-17be-4a76-9989-9390d8b7af84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:19.266108 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc31a5-17be-4a76-9989-9390d8b7af84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:19.268616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc31a5-17be-4a76-9989-9390d8b7af84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:19.271530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc31a5-17be-4a76-9989-9390d8b7af84 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:19.313025 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1073/2150] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:47:19.318258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b4b6a-c5ff-4a80-88c3-2f66fff920ac admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:19.337895 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b4b6a-c5ff-4a80-88c3-2f66fff920ac admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:19.347459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662df127-0f1c-4cfb-bce2-f5ce5716485b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:19.348731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662df127-0f1c-4cfb-bce2-f5ce5716485b admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:19.349336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662df127-0f1c-4cfb-bce2-f5ce5716485b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:19.350366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662df127-0f1c-4cfb-bce2-f5ce5716485b admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:19.352711 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662df127-0f1c-4cfb-bce2-f5ce5716485b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:19.356259 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662df127-0f1c-4cfb-bce2-f5ce5716485b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:19.383552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1078/2151] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:47:19.385766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc31a5-17be-4a76-9989-9390d8b7af84 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:19.420251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc31a5-17be-4a76-9989-9390d8b7af84 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:19.443710 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5c232-54c9-4f5e-b098-8f26b7edba0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:19.444971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5c232-54c9-4f5e-b098-8f26b7edba0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:19.445256 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5c232-54c9-4f5e-b098-8f26b7edba0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:19.445619 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5c232-54c9-4f5e-b098-8f26b7edba0a admin admin] PATH_INFO: `/v3/projects/d29d875fc4d84f92801112d6d2accd13/users/bf6b353833f14c3d819164e23374591e/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:19.476526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5c232-54c9-4f5e-b098-8f26b7edba0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d29d875fc4d84f92801112d6d2accd13, user_id=bf6b353833f14c3d819164e23374591e, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:19.481941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5c232-54c9-4f5e-b098-8f26b7edba0a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:19.510493 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1074/2152] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:47:19 2022] PUT /identity/v3/projects/d29d875fc4d84f92801112d6d2accd13/users/bf6b353833f14c3d819164e23374591e/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:19.516467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c3a8e-f3cd-4a44-a5fa-07ccb1767d0c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:19.516747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c3a8e-f3cd-4a44-a5fa-07ccb1767d0c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:19.517024 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c3a8e-f3cd-4a44-a5fa-07ccb1767d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:19.545180 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-de9c3a8e-f3cd-4a44-a5fa-07ccb1767d0c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:19.546452 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-de9c3a8e-f3cd-4a44-a5fa-07ccb1767d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:19.664768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1079/2153] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:47:19.714344 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5c232-54c9-4f5e-b098-8f26b7edba0a admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:19.717293 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-73e5c232-54c9-4f5e-b098-8f26b7edba0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:19.816594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5c232-54c9-4f5e-b098-8f26b7edba0a admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:19.823507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f5012-668a-4c27-ba25-313f7fd99762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:19.824625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f5012-668a-4c27-ba25-313f7fd99762 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:19.825852 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f5012-668a-4c27-ba25-313f7fd99762 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:19.825852 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f5012-668a-4c27-ba25-313f7fd99762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:19.843747 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f5012-668a-4c27-ba25-313f7fd99762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:19.846562 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f5012-668a-4c27-ba25-313f7fd99762 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:19.865841 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1075/2154] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:21.406832 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b228fb7-53c5-4bbb-ac52-34368cc57a36 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:21.407779 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b228fb7-53c5-4bbb-ac52-34368cc57a36 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:21.407779 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b228fb7-53c5-4bbb-ac52-34368cc57a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:21.455607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-5b228fb7-53c5-4bbb-ac52-34368cc57a36 None None] MFA Rules not processed for user `bf6b353833f14c3d819164e23374591e`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:21.459148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5b228fb7-53c5-4bbb-ac52-34368cc57a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:21.641565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1080/2155] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:47:21.653869 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f5012-668a-4c27-ba25-313f7fd99762 service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:21.655487 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-148f5012-668a-4c27-ba25-313f7fd99762 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:21.731348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:21.758456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f5012-668a-4c27-ba25-313f7fd99762 service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:21.770655 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632bc59-c071-4066-b2c1-68d6b02bd7eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:21.781532 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632bc59-c071-4066-b2c1-68d6b02bd7eb service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:21.781945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632bc59-c071-4066-b2c1-68d6b02bd7eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:21.782112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632bc59-c071-4066-b2c1-68d6b02bd7eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:21.797188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632bc59-c071-4066-b2c1-68d6b02bd7eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:21.798770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632bc59-c071-4066-b2c1-68d6b02bd7eb service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:21.814719 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1076/2156] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:23.667050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ec881-1c92-49b3-9f65-feb1e0fb705a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:23.668517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ec881-1c92-49b3-9f65-feb1e0fb705a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:23.669314 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ec881-1c92-49b3-9f65-feb1e0fb705a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:23.744738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-5b6ec881-1c92-49b3-9f65-feb1e0fb705a None None] MFA Rules not processed for user `bf6b353833f14c3d819164e23374591e`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:23.751162 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5b6ec881-1c92-49b3-9f65-feb1e0fb705a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:23.782339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1081/2157] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:25.090620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a808a1e-1a58-45e3-acaf-970ab1ce5188 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:25.099934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a808a1e-1a58-45e3-acaf-970ab1ce5188 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:25.100525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a808a1e-1a58-45e3-acaf-970ab1ce5188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:25.143247 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-6a808a1e-1a58-45e3-acaf-970ab1ce5188 None None] MFA Rules not processed for user `eb1aa3917d3d427e91f0805e5d6ab4eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:25.144781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6a808a1e-1a58-45e3-acaf-970ab1ce5188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:25.246076 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1077/2158] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:25.251455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ec881-1c92-49b3-9f65-feb1e0fb705a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:25.252989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5b6ec881-1c92-49b3-9f65-feb1e0fb705a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:25.265243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:25.280404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ec881-1c92-49b3-9f65-feb1e0fb705a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:25.288684 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6b3ea-4bf4-475a-96fd-9a4973627d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:25.289813 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6b3ea-4bf4-475a-96fd-9a4973627d24 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:25.290208 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6b3ea-4bf4-475a-96fd-9a4973627d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:25.290661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6b3ea-4bf4-475a-96fd-9a4973627d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:25.291511 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed6b3ea-4bf4-475a-96fd-9a4973627d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:25.294234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed6b3ea-4bf4-475a-96fd-9a4973627d24 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:25.327349 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1082/2159] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:47:25 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:47:25.331825 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a808a1e-1a58-45e3-acaf-970ab1ce5188 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:25.346741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a808a1e-1a58-45e3-acaf-970ab1ce5188 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:25.414832 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d750d43-7cbe-4100-bcd2-67050f0366ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:25.415466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d750d43-7cbe-4100-bcd2-67050f0366ab admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:25.415744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d750d43-7cbe-4100-bcd2-67050f0366ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:25.416041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d750d43-7cbe-4100-bcd2-67050f0366ab admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:25.416703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d750d43-7cbe-4100-bcd2-67050f0366ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:25.418777 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d750d43-7cbe-4100-bcd2-67050f0366ab admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:25.444548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1078/2160] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:47:25 2022] POST /identity/v3/users => generated 299 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:47:25.448832 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6b3ea-4bf4-475a-96fd-9a4973627d24 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:25.460775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6b3ea-4bf4-475a-96fd-9a4973627d24 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:25.465924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e89cb-6546-4279-a576-ecb97646db92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:25.466563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e89cb-6546-4279-a576-ecb97646db92 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:25.466865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e89cb-6546-4279-a576-ecb97646db92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:25.467089 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e89cb-6546-4279-a576-ecb97646db92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:25.467755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e89cb-6546-4279-a576-ecb97646db92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:25.469567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e89cb-6546-4279-a576-ecb97646db92 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:25.481224 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1083/2161] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:47:25.485538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d750d43-7cbe-4100-bcd2-67050f0366ab admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:25.496657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d750d43-7cbe-4100-bcd2-67050f0366ab admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:25.501831 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafee52-9c07-4d53-b21c-33d27642015a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:25.502531 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafee52-9c07-4d53-b21c-33d27642015a admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:25.502811 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafee52-9c07-4d53-b21c-33d27642015a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:25.503263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafee52-9c07-4d53-b21c-33d27642015a admin admin] PATH_INFO: `/v3/projects/9df09fbfd5084868a818e53fe4550329/users/aefb376ac5874039bce826475943fe15/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:25.521594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafee52-9c07-4d53-b21c-33d27642015a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df09fbfd5084868a818e53fe4550329, user_id=aefb376ac5874039bce826475943fe15, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:25.524738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafee52-9c07-4d53-b21c-33d27642015a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:25.548660 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1079/2162] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:47:25 2022] PUT /identity/v3/projects/9df09fbfd5084868a818e53fe4550329/users/aefb376ac5874039bce826475943fe15/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:47:25.560112 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e89cb-6546-4279-a576-ecb97646db92 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:25.572738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e89cb-6546-4279-a576-ecb97646db92 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:25.686236 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76059579-52e0-472e-8656-70fe3fd42162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:25.686774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76059579-52e0-472e-8656-70fe3fd42162 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:25.687053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76059579-52e0-472e-8656-70fe3fd42162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:25.687321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76059579-52e0-472e-8656-70fe3fd42162 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:25.687978 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76059579-52e0-472e-8656-70fe3fd42162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:25.689652 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76059579-52e0-472e-8656-70fe3fd42162 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:25.711251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1084/2163] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:47:25.716089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafee52-9c07-4d53-b21c-33d27642015a admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:25.729653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafee52-9c07-4d53-b21c-33d27642015a admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:25.737934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ffe33-728d-400f-9503-73dc3db0afe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:25.739046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ffe33-728d-400f-9503-73dc3db0afe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:25.739515 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ffe33-728d-400f-9503-73dc3db0afe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:25.739985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ffe33-728d-400f-9503-73dc3db0afe7 admin admin] PATH_INFO: `/v3/projects/9df09fbfd5084868a818e53fe4550329/users/aefb376ac5874039bce826475943fe15/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:25.742980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ffe33-728d-400f-9503-73dc3db0afe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df09fbfd5084868a818e53fe4550329, user_id=aefb376ac5874039bce826475943fe15, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:25.747019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ffe33-728d-400f-9503-73dc3db0afe7 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:25.763707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1080/2164] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:47:25 2022] PUT /identity/v3/projects/9df09fbfd5084868a818e53fe4550329/users/aefb376ac5874039bce826475943fe15/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:31.985234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76059579-52e0-472e-8656-70fe3fd42162 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:31.988178 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-76059579-52e0-472e-8656-70fe3fd42162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:32.008708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76059579-52e0-472e-8656-70fe3fd42162 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:32.133709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e11a4-048e-4511-8959-f077b94ff3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:32.135217 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e11a4-048e-4511-8959-f077b94ff3d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:32.135726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e11a4-048e-4511-8959-f077b94ff3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:32.136233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e11a4-048e-4511-8959-f077b94ff3d2 admin admin] PATH_INFO: `/v3/users/6e9a297e7be746e2bff3615033ca286e` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:32.139609 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e11a4-048e-4511-8959-f077b94ff3d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e9a297e7be746e2bff3615033ca286e)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:32.143389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e11a4-048e-4511-8959-f077b94ff3d2 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:32.266645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c78e11a4-048e-4511-8959-f077b94ff3d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e9a297e7be746e2bff3615033ca286e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:32.277027 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c78e11a4-048e-4511-8959-f077b94ff3d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e9a297e7be746e2bff3615033ca286e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:32.290230 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c78e11a4-048e-4511-8959-f077b94ff3d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e9a297e7be746e2bff3615033ca286e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:32.291330 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c78e11a4-048e-4511-8959-f077b94ff3d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e9a297e7be746e2bff3615033ca286e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:32.298616 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:32.311639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c78e11a4-048e-4511-8959-f077b94ff3d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e9a297e7be746e2bff3615033ca286e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:32.322923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1085/2165] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:47:31 2022] DELETE /identity/v3/users/6e9a297e7be746e2bff3615033ca286e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:32.328769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2359f-a24c-4411-9657-9cdc1601d176 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:32.330112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2359f-a24c-4411-9657-9cdc1601d176 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:32.330542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2359f-a24c-4411-9657-9cdc1601d176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:32.357847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-6bb2359f-a24c-4411-9657-9cdc1601d176 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:32.359116 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6bb2359f-a24c-4411-9657-9cdc1601d176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:32.398156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:32.447630 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1081/2166] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:32.459971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e11a4-048e-4511-8959-f077b94ff3d2 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:32.462215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c78e11a4-048e-4511-8959-f077b94ff3d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:32.537316 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e11a4-048e-4511-8959-f077b94ff3d2 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:32.550094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a7df6-3ed5-4d2f-82a2-ef79554b6a02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:32.550961 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a7df6-3ed5-4d2f-82a2-ef79554b6a02 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:32.551353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a7df6-3ed5-4d2f-82a2-ef79554b6a02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:32.551722 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a7df6-3ed5-4d2f-82a2-ef79554b6a02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:32.572847 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a7df6-3ed5-4d2f-82a2-ef79554b6a02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:32.572847 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a7df6-3ed5-4d2f-82a2-ef79554b6a02 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:32.593215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1086/2167] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:33.137765 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2359f-a24c-4411-9657-9cdc1601d176 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:33.139250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6bb2359f-a24c-4411-9657-9cdc1601d176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:33.159841 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2359f-a24c-4411-9657-9cdc1601d176 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:33.172769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b822d-c99d-4185-82dd-24db4358f22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:33.173648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b822d-c99d-4185-82dd-24db4358f22e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:33.173934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b822d-c99d-4185-82dd-24db4358f22e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:33.174258 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b822d-c99d-4185-82dd-24db4358f22e admin admin] PATH_INFO: `/v3/projects/1c11e40cb7554d419d54161d941a4760` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:33.175541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b822d-c99d-4185-82dd-24db4358f22e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c11e40cb7554d419d54161d941a4760)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:33.177763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b822d-c99d-4185-82dd-24db4358f22e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:33.265598 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a31b822d-c99d-4185-82dd-24db4358f22e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c11e40cb7554d419d54161d941a4760', 'request_id': 'req-a31b822d-c99d-4185-82dd-24db4358f22e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:33.290989 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a31b822d-c99d-4185-82dd-24db4358f22e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c11e40cb7554d419d54161d941a4760', 'request_id': 'req-a31b822d-c99d-4185-82dd-24db4358f22e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:33.305997 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a31b822d-c99d-4185-82dd-24db4358f22e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c11e40cb7554d419d54161d941a4760', 'request_id': 'req-a31b822d-c99d-4185-82dd-24db4358f22e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:33.346327 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a31b822d-c99d-4185-82dd-24db4358f22e admin admin] The token cache is being invalidate because project 1c11e40cb7554d419d54161d941a4760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:47:33.347272 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a31b822d-c99d-4185-82dd-24db4358f22e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:33.347485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a31b822d-c99d-4185-82dd-24db4358f22e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:33.349317 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1082/2168] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:47:33 2022] DELETE /identity/v3/projects/1c11e40cb7554d419d54161d941a4760 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:33.355781 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7e8d7-9c55-40ac-8bf3-be44a3dfada7 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:33.356422 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7e8d7-9c55-40ac-8bf3-be44a3dfada7 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:33.356422 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7e8d7-9c55-40ac-8bf3-be44a3dfada7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:33.391742 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-29d7e8d7-9c55-40ac-8bf3-be44a3dfada7 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:33.395134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-29d7e8d7-9c55-40ac-8bf3-be44a3dfada7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:33.483905 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1087/2169] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:33.491691 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28537efb-84fc-453f-ac7f-693e5046f84f None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:33.491982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28537efb-84fc-453f-ac7f-693e5046f84f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:33.492258 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28537efb-84fc-453f-ac7f-693e5046f84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:33.544851 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-28537efb-84fc-453f-ac7f-693e5046f84f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:33.546164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-28537efb-84fc-453f-ac7f-693e5046f84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:33.555949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1083/2170] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:33.560489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7e8d7-9c55-40ac-8bf3-be44a3dfada7 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:33.582845 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7e8d7-9c55-40ac-8bf3-be44a3dfada7 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:33.588828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dabb2-c9fc-4de9-97f6-59eb01a778ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:33.589487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dabb2-c9fc-4de9-97f6-59eb01a778ce admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:33.589788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dabb2-c9fc-4de9-97f6-59eb01a778ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:33.590039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dabb2-c9fc-4de9-97f6-59eb01a778ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:33.590712 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dabb2-c9fc-4de9-97f6-59eb01a778ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:47:33.591200 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dabb2-c9fc-4de9-97f6-59eb01a778ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:33.593069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dabb2-c9fc-4de9-97f6-59eb01a778ce admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:33.616782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1088/2171] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:47:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:47:33.622548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28537efb-84fc-453f-ac7f-693e5046f84f None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:33.636576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28537efb-84fc-453f-ac7f-693e5046f84f None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:33.642038 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb65436-8381-4e7f-9dd8-fc720d7a4f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:33.642729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb65436-8381-4e7f-9dd8-fc720d7a4f56 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:33.643009 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb65436-8381-4e7f-9dd8-fc720d7a4f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:33.643449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb65436-8381-4e7f-9dd8-fc720d7a4f56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:33.644661 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb65436-8381-4e7f-9dd8-fc720d7a4f56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:33.646815 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb65436-8381-4e7f-9dd8-fc720d7a4f56 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:33.696619 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1084/2172] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:47:33 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:47:33.699030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dabb2-c9fc-4de9-97f6-59eb01a778ce admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:33.713223 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dabb2-c9fc-4de9-97f6-59eb01a778ce admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:33.829576 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74f58a-30c6-47ca-bbf8-c04f7399a6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:33.830238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74f58a-30c6-47ca-bbf8-c04f7399a6b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:33.830510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74f58a-30c6-47ca-bbf8-c04f7399a6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:33.830772 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74f58a-30c6-47ca-bbf8-c04f7399a6b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:33.831738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74f58a-30c6-47ca-bbf8-c04f7399a6b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:33.833825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74f58a-30c6-47ca-bbf8-c04f7399a6b5 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:33.867968 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1089/2173] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:47:33 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:47:33.874420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb65436-8381-4e7f-9dd8-fc720d7a4f56 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:33.888720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb65436-8381-4e7f-9dd8-fc720d7a4f56 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:33.894422 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a69dd-a3d9-4b31-9ddc-d2251f2fbfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:33.895449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a69dd-a3d9-4b31-9ddc-d2251f2fbfe0 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:33.895955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a69dd-a3d9-4b31-9ddc-d2251f2fbfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:33.896357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a69dd-a3d9-4b31-9ddc-d2251f2fbfe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:33.897136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a69dd-a3d9-4b31-9ddc-d2251f2fbfe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:33.899402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a69dd-a3d9-4b31-9ddc-d2251f2fbfe0 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:33.915877 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1085/2174] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:47:33.922848 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74f58a-30c6-47ca-bbf8-c04f7399a6b5 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:33.940342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74f58a-30c6-47ca-bbf8-c04f7399a6b5 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:33.945279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd735c6-aba1-4904-afda-a061542b1aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:33.946184 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd735c6-aba1-4904-afda-a061542b1aad admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:33.946518 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd735c6-aba1-4904-afda-a061542b1aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:33.947682 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd735c6-aba1-4904-afda-a061542b1aad admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:33.947682 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd735c6-aba1-4904-afda-a061542b1aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:33.949660 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd735c6-aba1-4904-afda-a061542b1aad admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:33.951551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:33.973723 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1090/2175] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:47:33.977388 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a69dd-a3d9-4b31-9ddc-d2251f2fbfe0 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:33.989113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a69dd-a3d9-4b31-9ddc-d2251f2fbfe0 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:33.994325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f5b53-9f0e-4c5d-ac59-d720a49ea9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:33.995021 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f5b53-9f0e-4c5d-ac59-d720a49ea9d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:33.995294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f5b53-9f0e-4c5d-ac59-d720a49ea9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:33.995561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f5b53-9f0e-4c5d-ac59-d720a49ea9d5 admin admin] PATH_INFO: `/v3/projects/a5a499bf14ac41f892982cc3605b6edf/users/7e2c7f014f0742d6969d7baa0cb09e43/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:34.014402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f5b53-9f0e-4c5d-ac59-d720a49ea9d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a499bf14ac41f892982cc3605b6edf, user_id=7e2c7f014f0742d6969d7baa0cb09e43, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:34.018262 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f5b53-9f0e-4c5d-ac59-d720a49ea9d5 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:34.036761 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1086/2176] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:47:33 2022] PUT /identity/v3/projects/a5a499bf14ac41f892982cc3605b6edf/users/7e2c7f014f0742d6969d7baa0cb09e43/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:34.049565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1840eed-f07c-42c3-ae65-2f12c03cbab8 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:34.049988 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1840eed-f07c-42c3-ae65-2f12c03cbab8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:34.050323 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1840eed-f07c-42c3-ae65-2f12c03cbab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:34.126536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a1840eed-f07c-42c3-ae65-2f12c03cbab8 None None] MFA Rules not processed for user `7e2c7f014f0742d6969d7baa0cb09e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:34.127534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a1840eed-f07c-42c3-ae65-2f12c03cbab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:34.290422 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1091/2177] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:34.309734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f5b53-9f0e-4c5d-ac59-d720a49ea9d5 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:34.313403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-be0f5b53-9f0e-4c5d-ac59-d720a49ea9d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:34.378317 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:34.383151 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f5b53-9f0e-4c5d-ac59-d720a49ea9d5 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:34.391915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940a1c7-c732-4746-87f3-6abd8dbad0cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:34.393805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940a1c7-c732-4746-87f3-6abd8dbad0cd service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:34.393805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940a1c7-c732-4746-87f3-6abd8dbad0cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:34.394696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940a1c7-c732-4746-87f3-6abd8dbad0cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:34.413994 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940a1c7-c732-4746-87f3-6abd8dbad0cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:34.413994 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940a1c7-c732-4746-87f3-6abd8dbad0cd service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:34.436134 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1087/2178] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:34.463335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1840eed-f07c-42c3-ae65-2f12c03cbab8 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:34.573841 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1840eed-f07c-42c3-ae65-2f12c03cbab8 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:34.579798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e81c1-9f68-4f03-8b0d-cdd83ed99855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:34.580563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e81c1-9f68-4f03-8b0d-cdd83ed99855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:34.580915 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e81c1-9f68-4f03-8b0d-cdd83ed99855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:34.581137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e81c1-9f68-4f03-8b0d-cdd83ed99855 admin admin] PATH_INFO: `/v3/users/7e2c7f014f0742d6969d7baa0cb09e43` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:34.582372 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e81c1-9f68-4f03-8b0d-cdd83ed99855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e2c7f014f0742d6969d7baa0cb09e43)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:34.584482 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e81c1-9f68-4f03-8b0d-cdd83ed99855 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:34.668756 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-627e81c1-9f68-4f03-8b0d-cdd83ed99855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e2c7f014f0742d6969d7baa0cb09e43'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:34.683211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-627e81c1-9f68-4f03-8b0d-cdd83ed99855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e2c7f014f0742d6969d7baa0cb09e43'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:34.700256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-627e81c1-9f68-4f03-8b0d-cdd83ed99855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e2c7f014f0742d6969d7baa0cb09e43'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:34.701079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-627e81c1-9f68-4f03-8b0d-cdd83ed99855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e2c7f014f0742d6969d7baa0cb09e43'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:34.718516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-627e81c1-9f68-4f03-8b0d-cdd83ed99855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e2c7f014f0742d6969d7baa0cb09e43'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:34.731366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1092/2179] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:47:34 2022] DELETE /identity/v3/users/7e2c7f014f0742d6969d7baa0cb09e43 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:34.735834 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587edfaf-dfcd-446b-8c68-9d966cb3cf5c None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:34.736319 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587edfaf-dfcd-446b-8c68-9d966cb3cf5c None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:34.736595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587edfaf-dfcd-446b-8c68-9d966cb3cf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:34.762476 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-587edfaf-dfcd-446b-8c68-9d966cb3cf5c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:34.763619 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-587edfaf-dfcd-446b-8c68-9d966cb3cf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:34.869298 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1088/2180] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:34.883288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e81c1-9f68-4f03-8b0d-cdd83ed99855 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:34.885133 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-627e81c1-9f68-4f03-8b0d-cdd83ed99855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:34.896742 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:34.978449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e81c1-9f68-4f03-8b0d-cdd83ed99855 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:34.985325 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199b3ca-c697-4ead-a2ac-dfdd1db87b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:34.986593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199b3ca-c697-4ead-a2ac-dfdd1db87b6a service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:34.987045 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199b3ca-c697-4ead-a2ac-dfdd1db87b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:34.987381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199b3ca-c697-4ead-a2ac-dfdd1db87b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:34.999816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199b3ca-c697-4ead-a2ac-dfdd1db87b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:35.002175 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199b3ca-c697-4ead-a2ac-dfdd1db87b6a service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:35.016298 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1093/2181] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:35.376732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587edfaf-dfcd-446b-8c68-9d966cb3cf5c None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:35.380342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-587edfaf-dfcd-446b-8c68-9d966cb3cf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:35.407162 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587edfaf-dfcd-446b-8c68-9d966cb3cf5c None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:35.418640 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd8d6a-f29a-4592-b7a0-d2f87094bc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:35.419954 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd8d6a-f29a-4592-b7a0-d2f87094bc91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:35.420692 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd8d6a-f29a-4592-b7a0-d2f87094bc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:35.421360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd8d6a-f29a-4592-b7a0-d2f87094bc91 admin admin] PATH_INFO: `/v3/projects/a5a499bf14ac41f892982cc3605b6edf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:35.423897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd8d6a-f29a-4592-b7a0-d2f87094bc91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a499bf14ac41f892982cc3605b6edf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:35.426976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd8d6a-f29a-4592-b7a0-d2f87094bc91 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:35.443030 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d4423-ec35-4138-852a-3203d853cd70 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:35.443530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d4423-ec35-4138-852a-3203d853cd70 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:35.443969 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d4423-ec35-4138-852a-3203d853cd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:35.494901 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-502d4423-ec35-4138-852a-3203d853cd70 None None] MFA Rules not processed for user `aefb376ac5874039bce826475943fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:35.496916 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-502d4423-ec35-4138-852a-3203d853cd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:35.545018 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a4fd8d6a-f29a-4592-b7a0-d2f87094bc91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a499bf14ac41f892982cc3605b6edf', 'request_id': 'req-a4fd8d6a-f29a-4592-b7a0-d2f87094bc91'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:35.556362 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a4fd8d6a-f29a-4592-b7a0-d2f87094bc91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a499bf14ac41f892982cc3605b6edf', 'request_id': 'req-a4fd8d6a-f29a-4592-b7a0-d2f87094bc91'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:35.572882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a4fd8d6a-f29a-4592-b7a0-d2f87094bc91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a499bf14ac41f892982cc3605b6edf', 'request_id': 'req-a4fd8d6a-f29a-4592-b7a0-d2f87094bc91'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:35.592755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a4fd8d6a-f29a-4592-b7a0-d2f87094bc91 admin admin] The token cache is being invalidate because project a5a499bf14ac41f892982cc3605b6edf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:47:35.593213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a4fd8d6a-f29a-4592-b7a0-d2f87094bc91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:35.593964 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a4fd8d6a-f29a-4592-b7a0-d2f87094bc91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:35.595984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1089/2182] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:47:35 2022] DELETE /identity/v3/projects/a5a499bf14ac41f892982cc3605b6edf => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:35.601756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53da6ae-d57a-4798-ba2a-52f96e19e015 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:35.602208 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53da6ae-d57a-4798-ba2a-52f96e19e015 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:35.602650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53da6ae-d57a-4798-ba2a-52f96e19e015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:35.606559 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1094/2183] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:35.628163 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a53da6ae-d57a-4798-ba2a-52f96e19e015 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:35.628971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d4423-ec35-4138-852a-3203d853cd70 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:35.629429 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a53da6ae-d57a-4798-ba2a-52f96e19e015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:35.630668 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-502d4423-ec35-4138-852a-3203d853cd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:35.701140 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1090/2184] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:35.704700 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d4423-ec35-4138-852a-3203d853cd70 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:35.710280 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6765fb-58d2-4765-a042-65d1a69a6d5b None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:35.710654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6765fb-58d2-4765-a042-65d1a69a6d5b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:35.711197 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6765fb-58d2-4765-a042-65d1a69a6d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:35.738013 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-7f6765fb-58d2-4765-a042-65d1a69a6d5b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:35.739932 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7f6765fb-58d2-4765-a042-65d1a69a6d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:35.753578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1091/2185] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:35.756527 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6765fb-58d2-4765-a042-65d1a69a6d5b None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:35.763952 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:35.767437 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6765fb-58d2-4765-a042-65d1a69a6d5b None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:35.769613 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70e4b2-0960-4828-9144-e9c09d8c00af service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69de9368993f402cb6325e5022e064d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:35.770377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70e4b2-0960-4828-9144-e9c09d8c00af service glance] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:35.770725 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70e4b2-0960-4828-9144-e9c09d8c00af service glance] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:35.770981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70e4b2-0960-4828-9144-e9c09d8c00af service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:35.772130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93416a5-a8cc-4049-a804-e168fb49e1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:35.772814 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93416a5-a8cc-4049-a804-e168fb49e1f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:35.772942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ed70e4b2-0960-4828-9144-e9c09d8c00af service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:35.773092 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93416a5-a8cc-4049-a804-e168fb49e1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:35.773363 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93416a5-a8cc-4049-a804-e168fb49e1f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:35.774070 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93416a5-a8cc-4049-a804-e168fb49e1f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:47:35.774325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93416a5-a8cc-4049-a804-e168fb49e1f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:35.775983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93416a5-a8cc-4049-a804-e168fb49e1f5 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:35.784273 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:35.792899 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1092/2186] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:47:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:47:35.796156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93416a5-a8cc-4049-a804-e168fb49e1f5 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:35.808203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93416a5-a8cc-4049-a804-e168fb49e1f5 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:35.813196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2870119-8f40-4668-a9fb-1c375a8e4489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:35.813838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2870119-8f40-4668-a9fb-1c375a8e4489 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:35.814143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2870119-8f40-4668-a9fb-1c375a8e4489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:35.814355 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2870119-8f40-4668-a9fb-1c375a8e4489 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:35.814998 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2870119-8f40-4668-a9fb-1c375a8e4489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:35.816957 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2870119-8f40-4668-a9fb-1c375a8e4489 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:35.847001 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70e4b2-0960-4828-9144-e9c09d8c00af service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:35.851563 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1093/2187] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:47:35 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:47:35.851783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70e4b2-0960-4828-9144-e9c09d8c00af service glance] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:35.852953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2870119-8f40-4668-a9fb-1c375a8e4489 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:35.859043 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1095/2188] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:35.918846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2870119-8f40-4668-a9fb-1c375a8e4489 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:35.926905 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d8165-fc47-4394-b778-d3c0c2633acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:35.927725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d8165-fc47-4394-b778-d3c0c2633acb admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:35.928027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d8165-fc47-4394-b778-d3c0c2633acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:35.928342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d8165-fc47-4394-b778-d3c0c2633acb admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:35.929048 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d8165-fc47-4394-b778-d3c0c2633acb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:35.931403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d8165-fc47-4394-b778-d3c0c2633acb admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:35.961874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1094/2189] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:47:35 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:47:35.966074 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70e4b2-0960-4828-9144-e9c09d8c00af service glance] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:35.979985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70e4b2-0960-4828-9144-e9c09d8c00af service glance] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:35.985066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a370ed1-52af-4578-8ea7-8d9bbe89a1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:35.985639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a370ed1-52af-4578-8ea7-8d9bbe89a1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:35.985977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a370ed1-52af-4578-8ea7-8d9bbe89a1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:35.986258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a370ed1-52af-4578-8ea7-8d9bbe89a1a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:35.986913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a370ed1-52af-4578-8ea7-8d9bbe89a1a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:35.988865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a370ed1-52af-4578-8ea7-8d9bbe89a1a2 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:35.990541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc61595-3d7f-4c52-8068-db350a721ad3 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:35.990939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc61595-3d7f-4c52-8068-db350a721ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:35.991356 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc61595-3d7f-4c52-8068-db350a721ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:36.004061 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1096/2190] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:47:36.006038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a370ed1-52af-4578-8ea7-8d9bbe89a1a2 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:36.018386 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a370ed1-52af-4578-8ea7-8d9bbe89a1a2 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:36.018852 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-7bc61595-3d7f-4c52-8068-db350a721ad3 None None] MFA Rules not processed for user `8fc940594a1848aa9644b645049a1bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:36.021593 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7bc61595-3d7f-4c52-8068-db350a721ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:36.027233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0a791-b033-4712-b187-a4792d54e1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:36.027957 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0a791-b033-4712-b187-a4792d54e1ef admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:36.028317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0a791-b033-4712-b187-a4792d54e1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:36.028587 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0a791-b033-4712-b187-a4792d54e1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:36.029259 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0a791-b033-4712-b187-a4792d54e1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:36.030949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0a791-b033-4712-b187-a4792d54e1ef admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:36.051310 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1097/2191] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:47:36.057909 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0a791-b033-4712-b187-a4792d54e1ef admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:36.078918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0a791-b033-4712-b187-a4792d54e1ef admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:36.086099 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2516d-03fc-4bf9-8752-6ef4d05de3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:36.087456 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2516d-03fc-4bf9-8752-6ef4d05de3d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:36.088403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2516d-03fc-4bf9-8752-6ef4d05de3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:36.088684 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2516d-03fc-4bf9-8752-6ef4d05de3d9 admin admin] PATH_INFO: `/v3/projects/f2c2350f7bf04d7b9e490693a925c73a/users/98dc1421ff3b4b5da5e31e50f480edb5/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:36.125401 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1095/2192] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Aug 30 14:47:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:47:36.131597 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2516d-03fc-4bf9-8752-6ef4d05de3d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2c2350f7bf04d7b9e490693a925c73a, user_id=98dc1421ff3b4b5da5e31e50f480edb5, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:36.138011 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2516d-03fc-4bf9-8752-6ef4d05de3d9 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:36.141117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc61595-3d7f-4c52-8068-db350a721ad3 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:36.142620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7bc61595-3d7f-4c52-8068-db350a721ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:36.165717 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1098/2193] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:47:36 2022] PUT /identity/v3/projects/f2c2350f7bf04d7b9e490693a925c73a/users/98dc1421ff3b4b5da5e31e50f480edb5/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:36.176695 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4613c9-ebbd-4140-af2c-70d458a03770 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:36.177075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4613c9-ebbd-4140-af2c-70d458a03770 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:36.177793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4613c9-ebbd-4140-af2c-70d458a03770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:36.242801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ae4613c9-ebbd-4140-af2c-70d458a03770 None None] MFA Rules not processed for user `98dc1421ff3b4b5da5e31e50f480edb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:36.245876 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ae4613c9-ebbd-4140-af2c-70d458a03770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:36.283861 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc61595-3d7f-4c52-8068-db350a721ad3 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:36.299426 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:36.384429 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1099/2194] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:36.396856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4613c9-ebbd-4140-af2c-70d458a03770 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:36.398526 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ae4613c9-ebbd-4140-af2c-70d458a03770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:36.417271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365c38b-c84d-4c90-90e1-353242769287 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d98f5adfaf410580481417e7ba1638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:36.418703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365c38b-c84d-4c90-90e1-353242769287 service swift] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:36.419250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365c38b-c84d-4c90-90e1-353242769287 service swift] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:36.419795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365c38b-c84d-4c90-90e1-353242769287 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:36.494627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:36.509385 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4613c9-ebbd-4140-af2c-70d458a03770 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:36.516826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e365c38b-c84d-4c90-90e1-353242769287 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:36.518660 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e365c38b-c84d-4c90-90e1-353242769287 service swift] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:36.520816 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601ccc1-c212-4b2c-9083-c55b9fa8bd87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:36.521343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601ccc1-c212-4b2c-9083-c55b9fa8bd87 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:36.521637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601ccc1-c212-4b2c-9083-c55b9fa8bd87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:36.521894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601ccc1-c212-4b2c-9083-c55b9fa8bd87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:36.530948 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1096/2195] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Aug 30 14:47:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:47:36.544944 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601ccc1-c212-4b2c-9083-c55b9fa8bd87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:36.546607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601ccc1-c212-4b2c-9083-c55b9fa8bd87 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:36.563065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1100/2196] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:47:36.626806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365c38b-c84d-4c90-90e1-353242769287 service swift] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:36.632203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e365c38b-c84d-4c90-90e1-353242769287 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:36.893439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365c38b-c84d-4c90-90e1-353242769287 service swift] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:36.902801 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4c451-5025-48ef-ab2f-e1b3cbe66df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:36.903755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4c451-5025-48ef-ab2f-e1b3cbe66df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:36.904217 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4c451-5025-48ef-ab2f-e1b3cbe66df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:36.904656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4c451-5025-48ef-ab2f-e1b3cbe66df8 admin admin] PATH_INFO: `/v3/users/23fe3372d1cd4d2380b22fedd5636612` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:36.924912 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4c451-5025-48ef-ab2f-e1b3cbe66df8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23fe3372d1cd4d2380b22fedd5636612)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:36.927202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4c451-5025-48ef-ab2f-e1b3cbe66df8 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:37.082799 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6ac4c451-5025-48ef-ab2f-e1b3cbe66df8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23fe3372d1cd4d2380b22fedd5636612'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:37.084651 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6ac4c451-5025-48ef-ab2f-e1b3cbe66df8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23fe3372d1cd4d2380b22fedd5636612'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:37.096085 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6ac4c451-5025-48ef-ab2f-e1b3cbe66df8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23fe3372d1cd4d2380b22fedd5636612'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:37.106250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6ac4c451-5025-48ef-ab2f-e1b3cbe66df8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23fe3372d1cd4d2380b22fedd5636612'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:37.107963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:37.154159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6ac4c451-5025-48ef-ab2f-e1b3cbe66df8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23fe3372d1cd4d2380b22fedd5636612'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:37.169147 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1097/2197] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:47:36 2022] DELETE /identity/v3/users/23fe3372d1cd4d2380b22fedd5636612 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:37.173689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bae3e-1812-4dce-8296-4cd56e669a80 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:37.173976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bae3e-1812-4dce-8296-4cd56e669a80 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:37.174259 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bae3e-1812-4dce-8296-4cd56e669a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:37.202217 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c11bae3e-1812-4dce-8296-4cd56e669a80 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:37.203763 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c11bae3e-1812-4dce-8296-4cd56e669a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:37.289008 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1101/2198] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:47:37.295096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4c451-5025-48ef-ab2f-e1b3cbe66df8 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:37.297334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6ac4c451-5025-48ef-ab2f-e1b3cbe66df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:37.442440 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4c451-5025-48ef-ab2f-e1b3cbe66df8 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:37.448316 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b43f70-0b6a-469f-bce8-58b716b7b64b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:37.449641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b43f70-0b6a-469f-bce8-58b716b7b64b service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:37.450150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b43f70-0b6a-469f-bce8-58b716b7b64b service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:37.450584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b43f70-0b6a-469f-bce8-58b716b7b64b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:37.487861 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b43f70-0b6a-469f-bce8-58b716b7b64b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:37.491536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b43f70-0b6a-469f-bce8-58b716b7b64b service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:37.506940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1098/2199] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:37.991305 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bae3e-1812-4dce-8296-4cd56e669a80 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:37.993568 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c11bae3e-1812-4dce-8296-4cd56e669a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:38.054527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bae3e-1812-4dce-8296-4cd56e669a80 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:38.065708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048bffb-a656-4bfd-9fd0-83642ccf676a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:38.066484 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048bffb-a656-4bfd-9fd0-83642ccf676a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:38.066857 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048bffb-a656-4bfd-9fd0-83642ccf676a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:38.067227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048bffb-a656-4bfd-9fd0-83642ccf676a admin admin] PATH_INFO: `/v3/projects/084892e1b0834abba4f84624cae77379` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:38.069707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048bffb-a656-4bfd-9fd0-83642ccf676a admin admin] RBAC: Authorizing `identity:delete_project(project_id=084892e1b0834abba4f84624cae77379)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:38.080074 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048bffb-a656-4bfd-9fd0-83642ccf676a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:38.232769 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f048bffb-a656-4bfd-9fd0-83642ccf676a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '084892e1b0834abba4f84624cae77379', 'request_id': 'req-f048bffb-a656-4bfd-9fd0-83642ccf676a'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:38.247766 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f048bffb-a656-4bfd-9fd0-83642ccf676a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '084892e1b0834abba4f84624cae77379', 'request_id': 'req-f048bffb-a656-4bfd-9fd0-83642ccf676a'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:38.265454 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f048bffb-a656-4bfd-9fd0-83642ccf676a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '084892e1b0834abba4f84624cae77379', 'request_id': 'req-f048bffb-a656-4bfd-9fd0-83642ccf676a'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:38.293340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f048bffb-a656-4bfd-9fd0-83642ccf676a admin admin] The token cache is being invalidate because project 084892e1b0834abba4f84624cae77379 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:47:38.293637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f048bffb-a656-4bfd-9fd0-83642ccf676a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:38.296235 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f048bffb-a656-4bfd-9fd0-83642ccf676a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:38.305063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1102/2200] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:47:37 2022] DELETE /identity/v3/projects/084892e1b0834abba4f84624cae77379 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:38.306149 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a406ea-0afd-4f6b-bf97-efde68926f9e None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:38.306424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a406ea-0afd-4f6b-bf97-efde68926f9e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:38.306715 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a406ea-0afd-4f6b-bf97-efde68926f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:38.342498 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-08a406ea-0afd-4f6b-bf97-efde68926f9e None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:38.343752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-08a406ea-0afd-4f6b-bf97-efde68926f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:38.416099 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1099/2201] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:38.426871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8ee45-7425-4d11-9285-ee3ce1268036 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:38.427120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8ee45-7425-4d11-9285-ee3ce1268036 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:38.427409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8ee45-7425-4d11-9285-ee3ce1268036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:38.462676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-dcf8ee45-7425-4d11-9285-ee3ce1268036 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:38.464550 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-dcf8ee45-7425-4d11-9285-ee3ce1268036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:38.500221 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a406ea-0afd-4f6b-bf97-efde68926f9e None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:38.500960 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1103/2202] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:38.511394 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a406ea-0afd-4f6b-bf97-efde68926f9e None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:38.519713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bad144-5c63-4a26-a706-3022dfd7d57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:38.519713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bad144-5c63-4a26-a706-3022dfd7d57d admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:38.519713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bad144-5c63-4a26-a706-3022dfd7d57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:38.519713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bad144-5c63-4a26-a706-3022dfd7d57d admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:38.520213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bad144-5c63-4a26-a706-3022dfd7d57d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:47:38.520213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bad144-5c63-4a26-a706-3022dfd7d57d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:38.521167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bad144-5c63-4a26-a706-3022dfd7d57d admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:38.543236 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8ee45-7425-4d11-9285-ee3ce1268036 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:38.548815 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1100/2203] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:47:38.554588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8ee45-7425-4d11-9285-ee3ce1268036 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:38.560455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce775e-4be7-40dd-ac73-bbfb0a191591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:38.561093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce775e-4be7-40dd-ac73-bbfb0a191591 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:38.561467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce775e-4be7-40dd-ac73-bbfb0a191591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:38.561745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce775e-4be7-40dd-ac73-bbfb0a191591 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:38.563787 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce775e-4be7-40dd-ac73-bbfb0a191591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:38.564783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce775e-4be7-40dd-ac73-bbfb0a191591 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:38.732860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bad144-5c63-4a26-a706-3022dfd7d57d admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:38.733004 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1104/2204] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:47:38 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:47:38.749263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bad144-5c63-4a26-a706-3022dfd7d57d admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:38.963187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6660f3-ee13-4bcc-887b-764f83d5655c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:38.964341 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6660f3-ee13-4bcc-887b-764f83d5655c admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:38.965101 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6660f3-ee13-4bcc-887b-764f83d5655c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:38.965656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6660f3-ee13-4bcc-887b-764f83d5655c admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:38.966672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6660f3-ee13-4bcc-887b-764f83d5655c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:38.969626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6660f3-ee13-4bcc-887b-764f83d5655c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:38.984861 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:39.011622 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1101/2205] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:47:38 2022] POST /identity/v3/users => generated 289 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:47:39.021063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce775e-4be7-40dd-ac73-bbfb0a191591 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:39.050830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce775e-4be7-40dd-ac73-bbfb0a191591 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:39.064560 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26265175-32c3-4b5c-8fc4-10b01855e653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:39.065461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26265175-32c3-4b5c-8fc4-10b01855e653 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:39.065818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26265175-32c3-4b5c-8fc4-10b01855e653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:39.068349 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26265175-32c3-4b5c-8fc4-10b01855e653 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:39.069497 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26265175-32c3-4b5c-8fc4-10b01855e653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:39.072945 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26265175-32c3-4b5c-8fc4-10b01855e653 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:39.103436 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1105/2206] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:47:39.108288 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6660f3-ee13-4bcc-887b-764f83d5655c admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:39.136088 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6660f3-ee13-4bcc-887b-764f83d5655c admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:39.158139 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e750a-4157-48e1-a460-a0d24ff48529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:39.159171 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e750a-4157-48e1-a460-a0d24ff48529 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:39.159635 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e750a-4157-48e1-a460-a0d24ff48529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:39.160128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e750a-4157-48e1-a460-a0d24ff48529 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:39.161345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e750a-4157-48e1-a460-a0d24ff48529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:39.170838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e750a-4157-48e1-a460-a0d24ff48529 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:39.226168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26265175-32c3-4b5c-8fc4-10b01855e653 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:39.227295 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1102/2207] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:47:39.271731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26265175-32c3-4b5c-8fc4-10b01855e653 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:39.320017 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc7eb4-a531-4399-9d0f-86d46a6360f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:39.321832 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc7eb4-a531-4399-9d0f-86d46a6360f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:39.327413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc7eb4-a531-4399-9d0f-86d46a6360f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:39.329770 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc7eb4-a531-4399-9d0f-86d46a6360f6 admin admin] PATH_INFO: `/v3/projects/86111e1d9f5b485abf57231b21b3a5e9/users/06ab23cfa18e47c9871c6f451b871ef2/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:39.373753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc7eb4-a531-4399-9d0f-86d46a6360f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86111e1d9f5b485abf57231b21b3a5e9, user_id=06ab23cfa18e47c9871c6f451b871ef2, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:39.380056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc7eb4-a531-4399-9d0f-86d46a6360f6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:39.422355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1106/2208] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:47:39 2022] PUT /identity/v3/projects/86111e1d9f5b485abf57231b21b3a5e9/users/06ab23cfa18e47c9871c6f451b871ef2/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:39.439056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adeba5-04d4-4118-a518-5a15f949b551 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:39.439220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adeba5-04d4-4118-a518-5a15f949b551 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:39.439498 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adeba5-04d4-4118-a518-5a15f949b551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:39.478064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a7adeba5-04d4-4118-a518-5a15f949b551 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:39.481454 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a7adeba5-04d4-4118-a518-5a15f949b551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:39.607955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1103/2209] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:39.620108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc7eb4-a531-4399-9d0f-86d46a6360f6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:39.622674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e6dc7eb4-a531-4399-9d0f-86d46a6360f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:39.643097 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:39.754848 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc7eb4-a531-4399-9d0f-86d46a6360f6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:39.760802 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b70965-af4f-4352-b78b-79f2c0ccf70e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:39.762100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b70965-af4f-4352-b78b-79f2c0ccf70e service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:39.762842 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b70965-af4f-4352-b78b-79f2c0ccf70e service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:39.763327 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b70965-af4f-4352-b78b-79f2c0ccf70e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:39.785006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b70965-af4f-4352-b78b-79f2c0ccf70e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:39.787191 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b70965-af4f-4352-b78b-79f2c0ccf70e service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:39.801649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1107/2210] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:45.011522 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb3a56-c737-465f-8227-bb5d26ebcdac None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:45.012549 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb3a56-c737-465f-8227-bb5d26ebcdac None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:45.012988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb3a56-c737-465f-8227-bb5d26ebcdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:45.066787 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-2aeb3a56-c737-465f-8227-bb5d26ebcdac None None] MFA Rules not processed for user `3398704b0dbc4af08e94b774a88851a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:45.068041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2aeb3a56-c737-465f-8227-bb5d26ebcdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:45.167741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1104/2211] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:48.689257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23c98e-ea06-4a16-b5fa-ccaa56781ad4 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:48.690109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23c98e-ea06-4a16-b5fa-ccaa56781ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:48.690109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23c98e-ea06-4a16-b5fa-ccaa56781ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:48.803903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7b23c98e-ea06-4a16-b5fa-ccaa56781ad4 None None] MFA Rules not processed for user `06ab23cfa18e47c9871c6f451b871ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:48.811561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7b23c98e-ea06-4a16-b5fa-ccaa56781ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:49.040091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1108/2212] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:49.048378 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb3a56-c737-465f-8227-bb5d26ebcdac None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:49.090459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb3a56-c737-465f-8227-bb5d26ebcdac None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:49.151276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cda601-c94b-4a93-82da-4f5f65af2ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:49.154756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cda601-c94b-4a93-82da-4f5f65af2ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:49.155434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cda601-c94b-4a93-82da-4f5f65af2ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:49.156370 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cda601-c94b-4a93-82da-4f5f65af2ac2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:49.159681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cda601-c94b-4a93-82da-4f5f65af2ac2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:49.162978 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cda601-c94b-4a93-82da-4f5f65af2ac2 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:49.305467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23c98e-ea06-4a16-b5fa-ccaa56781ad4 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:49.317907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1105/2213] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:47:49 2022] POST /identity/v3/projects => generated 379 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:47:49.323393 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23c98e-ea06-4a16-b5fa-ccaa56781ad4 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:49.418598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653adb5d-681d-4196-9067-f4cacb9aa1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:49.418598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653adb5d-681d-4196-9067-f4cacb9aa1ff admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:49.418598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653adb5d-681d-4196-9067-f4cacb9aa1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:49.419118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653adb5d-681d-4196-9067-f4cacb9aa1ff admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:49.419937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653adb5d-681d-4196-9067-f4cacb9aa1ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:49.422060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653adb5d-681d-4196-9067-f4cacb9aa1ff admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:49.474784 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cda601-c94b-4a93-82da-4f5f65af2ac2 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:49.483851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1109/2214] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:47:49 2022] POST /identity/v3/users => generated 296 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:47:49.488623 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cda601-c94b-4a93-82da-4f5f65af2ac2 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:49.514297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2a210-d9fb-45a5-a9d2-7ab9a3768a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:49.517889 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2a210-d9fb-45a5-a9d2-7ab9a3768a41 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:49.518198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2a210-d9fb-45a5-a9d2-7ab9a3768a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:49.518473 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2a210-d9fb-45a5-a9d2-7ab9a3768a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:49.520690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2a210-d9fb-45a5-a9d2-7ab9a3768a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:49.524627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2a210-d9fb-45a5-a9d2-7ab9a3768a41 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:49.565871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653adb5d-681d-4196-9067-f4cacb9aa1ff admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:49.574552 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1106/2215] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:47:49.587585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653adb5d-681d-4196-9067-f4cacb9aa1ff admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:49.603973 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed79739-2738-42f0-9841-fa1cd62baac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:49.604838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed79739-2738-42f0-9841-fa1cd62baac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:49.605215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed79739-2738-42f0-9841-fa1cd62baac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:49.606144 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed79739-2738-42f0-9841-fa1cd62baac7 admin admin] PATH_INFO: `/v3/projects/e5f81b194ec04f058d62723478ba875a/users/b0353dcc5d924fc2bb594d2d941ee40b/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:49.675789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed79739-2738-42f0-9841-fa1cd62baac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5f81b194ec04f058d62723478ba875a, user_id=b0353dcc5d924fc2bb594d2d941ee40b, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:49.689130 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed79739-2738-42f0-9841-fa1cd62baac7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:49.741117 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1110/2216] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:47:49 2022] PUT /identity/v3/projects/e5f81b194ec04f058d62723478ba875a/users/b0353dcc5d924fc2bb594d2d941ee40b/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:49.749013 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2a210-d9fb-45a5-a9d2-7ab9a3768a41 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:49.775126 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2a210-d9fb-45a5-a9d2-7ab9a3768a41 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:49.891644 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:49.929316 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e04e23-2173-4203-9b8a-616a17452ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:49.933085 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e04e23-2173-4203-9b8a-616a17452ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:49.933369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e04e23-2173-4203-9b8a-616a17452ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:49.934009 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e04e23-2173-4203-9b8a-616a17452ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:49.936909 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e04e23-2173-4203-9b8a-616a17452ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:49.938744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e04e23-2173-4203-9b8a-616a17452ee8 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:49.978968 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1107/2217] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:47:49.983450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed79739-2738-42f0-9841-fa1cd62baac7 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:50.014517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed79739-2738-42f0-9841-fa1cd62baac7 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:50.054829 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602604f9-361d-4755-b12d-6f6cca494bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:50.055562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602604f9-361d-4755-b12d-6f6cca494bef admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:50.055852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602604f9-361d-4755-b12d-6f6cca494bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:50.059329 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602604f9-361d-4755-b12d-6f6cca494bef admin admin] PATH_INFO: `/v3/projects/e5f81b194ec04f058d62723478ba875a/users/b0353dcc5d924fc2bb594d2d941ee40b/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:50.093062 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602604f9-361d-4755-b12d-6f6cca494bef admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5f81b194ec04f058d62723478ba875a, user_id=b0353dcc5d924fc2bb594d2d941ee40b, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:50.095852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602604f9-361d-4755-b12d-6f6cca494bef admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:50.105350 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:50.137046 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1111/2218] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:47:49 2022] PUT /identity/v3/projects/e5f81b194ec04f058d62723478ba875a/users/b0353dcc5d924fc2bb594d2d941ee40b/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:51.756056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e04e23-2173-4203-9b8a-616a17452ee8 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:51.761878 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-53e04e23-2173-4203-9b8a-616a17452ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:51.839102 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e04e23-2173-4203-9b8a-616a17452ee8 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:52.066488 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e82ba7-eca9-480e-9366-bf071cd3f30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:52.068326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e82ba7-eca9-480e-9366-bf071cd3f30e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:52.069783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e82ba7-eca9-480e-9366-bf071cd3f30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:52.072228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e82ba7-eca9-480e-9366-bf071cd3f30e admin admin] PATH_INFO: `/v3/users/98dc1421ff3b4b5da5e31e50f480edb5` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:52.073988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e82ba7-eca9-480e-9366-bf071cd3f30e admin admin] RBAC: Authorizing `identity:delete_user(user_id=98dc1421ff3b4b5da5e31e50f480edb5)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:52.077230 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e82ba7-eca9-480e-9366-bf071cd3f30e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:52.330324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-29e82ba7-eca9-480e-9366-bf071cd3f30e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98dc1421ff3b4b5da5e31e50f480edb5'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:52.332389 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-29e82ba7-eca9-480e-9366-bf071cd3f30e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98dc1421ff3b4b5da5e31e50f480edb5'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:52.335833 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-29e82ba7-eca9-480e-9366-bf071cd3f30e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98dc1421ff3b4b5da5e31e50f480edb5'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:52.379814 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-29e82ba7-eca9-480e-9366-bf071cd3f30e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98dc1421ff3b4b5da5e31e50f480edb5'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:52.432733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-29e82ba7-eca9-480e-9366-bf071cd3f30e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98dc1421ff3b4b5da5e31e50f480edb5'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:52.496229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1108/2219] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:47:51 2022] DELETE /identity/v3/users/98dc1421ff3b4b5da5e31e50f480edb5 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:52.498962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d27daf-dec2-4c55-aba2-493de31a2eb9 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:52.499788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d27daf-dec2-4c55-aba2-493de31a2eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:52.499788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d27daf-dec2-4c55-aba2-493de31a2eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:52.540049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c4d27daf-dec2-4c55-aba2-493de31a2eb9 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:52.544538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c4d27daf-dec2-4c55-aba2-493de31a2eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:52.755042 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1112/2220] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:52.783673 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e82ba7-eca9-480e-9366-bf071cd3f30e admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:52.792795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-29e82ba7-eca9-480e-9366-bf071cd3f30e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:53.001296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e82ba7-eca9-480e-9366-bf071cd3f30e admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:53.010582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7914e-d6ad-4265-af6a-3dd63753e6e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:53.012263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7914e-d6ad-4265-af6a-3dd63753e6e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:53.013152 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7914e-d6ad-4265-af6a-3dd63753e6e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:53.013953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7914e-d6ad-4265-af6a-3dd63753e6e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:53.057207 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7914e-d6ad-4265-af6a-3dd63753e6e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:53.062968 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7914e-d6ad-4265-af6a-3dd63753e6e8 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:53.121952 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1109/2221] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:54.507307 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d27daf-dec2-4c55-aba2-493de31a2eb9 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:54.510782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c4d27daf-dec2-4c55-aba2-493de31a2eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:54.563459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d27daf-dec2-4c55-aba2-493de31a2eb9 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:54.592664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6b17b-7468-46cd-a853-e86119d432b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:54.593360 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6b17b-7468-46cd-a853-e86119d432b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:54.593638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6b17b-7468-46cd-a853-e86119d432b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:54.593903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6b17b-7468-46cd-a853-e86119d432b6 admin admin] PATH_INFO: `/v3/projects/f2c2350f7bf04d7b9e490693a925c73a` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:54.597909 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6b17b-7468-46cd-a853-e86119d432b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2c2350f7bf04d7b9e490693a925c73a)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:54.606003 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6b17b-7468-46cd-a853-e86119d432b6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:54.839551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a5b6b17b-7468-46cd-a853-e86119d432b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2c2350f7bf04d7b9e490693a925c73a', 'request_id': 'req-a5b6b17b-7468-46cd-a853-e86119d432b6'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:54.887018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a5b6b17b-7468-46cd-a853-e86119d432b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2c2350f7bf04d7b9e490693a925c73a', 'request_id': 'req-a5b6b17b-7468-46cd-a853-e86119d432b6'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:54.911145 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a5b6b17b-7468-46cd-a853-e86119d432b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2c2350f7bf04d7b9e490693a925c73a', 'request_id': 'req-a5b6b17b-7468-46cd-a853-e86119d432b6'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:54.964538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a5b6b17b-7468-46cd-a853-e86119d432b6 admin admin] The token cache is being invalidate because project f2c2350f7bf04d7b9e490693a925c73a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:47:54.965059 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a5b6b17b-7468-46cd-a853-e86119d432b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:54.966498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a5b6b17b-7468-46cd-a853-e86119d432b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:47:54.984041 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1113/2222] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:47:54 2022] DELETE /identity/v3/projects/f2c2350f7bf04d7b9e490693a925c73a => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:47:59.721162 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb3b02-d7a6-43fc-a57e-e08e6b3bf4f4 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:59.722133 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb3b02-d7a6-43fc-a57e-e08e6b3bf4f4 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:59.722571 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb3b02-d7a6-43fc-a57e-e08e6b3bf4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:59.755920 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-07eb3b02-d7a6-43fc-a57e-e08e6b3bf4f4 None None] MFA Rules not processed for user `eb1aa3917d3d427e91f0805e5d6ab4eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:59.757313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-07eb3b02-d7a6-43fc-a57e-e08e6b3bf4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:59.771476 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:59.878305 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1110/2223] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:47:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:48:00.209743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0fc29-9852-44ff-a7ab-e341aec21f55 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:00.210570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0fc29-9852-44ff-a7ab-e341aec21f55 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:00.210570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0fc29-9852-44ff-a7ab-e341aec21f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:00.266390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0ba0fc29-9852-44ff-a7ab-e341aec21f55 None None] MFA Rules not processed for user `b0353dcc5d924fc2bb594d2d941ee40b`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:48:00.267941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0ba0fc29-9852-44ff-a7ab-e341aec21f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:00.417967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1114/2224] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:48:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 30 14:48:00.442287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb3b02-d7a6-43fc-a57e-e08e6b3bf4f4 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:48:00.447187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-07eb3b02-d7a6-43fc-a57e-e08e6b3bf4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:00.614223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb3b02-d7a6-43fc-a57e-e08e6b3bf4f4 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:48:00.621633 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565777b-cf62-4400-8adc-beacf8450740 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:48:00.622663 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565777b-cf62-4400-8adc-beacf8450740 service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:00.623069 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565777b-cf62-4400-8adc-beacf8450740 service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:00.623475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565777b-cf62-4400-8adc-beacf8450740 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:00.626035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5565777b-cf62-4400-8adc-beacf8450740 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:00.700049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565777b-cf62-4400-8adc-beacf8450740 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:48:00.701933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565777b-cf62-4400-8adc-beacf8450740 service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:48:00.718267 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1111/2225] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:48:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:48:04.470956 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e38b85-0927-47b7-9dcf-323db9069463 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:04.472810 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e38b85-0927-47b7-9dcf-323db9069463 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:04.473321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e38b85-0927-47b7-9dcf-323db9069463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:04.493838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:04.527853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-06e38b85-0927-47b7-9dcf-323db9069463 None None] MFA Rules not processed for user `8fc940594a1848aa9644b645049a1bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:48:04.541738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-06e38b85-0927-47b7-9dcf-323db9069463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:04.741610 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1115/2226] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Aug 30 14:48:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:48:04.781477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565777b-cf62-4400-8adc-beacf8450740 service nova] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:48:04.784353 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5565777b-cf62-4400-8adc-beacf8450740 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:04.880566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:04.907103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565777b-cf62-4400-8adc-beacf8450740 service nova] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:48:04.916678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f478dfe-57bc-45c4-a1f2-8c8647719e79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d98f5adfaf410580481417e7ba1638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:48:04.918676 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f478dfe-57bc-45c4-a1f2-8c8647719e79 service swift] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:04.919330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f478dfe-57bc-45c4-a1f2-8c8647719e79 service swift] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:04.920028 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f478dfe-57bc-45c4-a1f2-8c8647719e79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:04.936398 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f478dfe-57bc-45c4-a1f2-8c8647719e79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:48:04.942100 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f478dfe-57bc-45c4-a1f2-8c8647719e79 service swift] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:48:04.967499 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1112/2227] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Aug 30 14:48:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 14:48:14.721868 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=24 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:14.955939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=8 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:38.436518 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dab6a-fd52-401b-9ba4-f1d50351f3bb None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:38.437051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dab6a-fd52-401b-9ba4-f1d50351f3bb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:38.437051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dab6a-fd52-401b-9ba4-f1d50351f3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:38.456527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1c5dab6a-fd52-401b-9ba4-f1d50351f3bb None None] MFA Rules not processed for user `06ab23cfa18e47c9871c6f451b871ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:48:38.458125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1c5dab6a-fd52-401b-9ba4-f1d50351f3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:38.466002 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1116/2228] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:48:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:48:48.450569 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:03.506652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f478dfe-57bc-45c4-a1f2-8c8647719e79 service swift] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:03.508166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-3f478dfe-57bc-45c4-a1f2-8c8647719e79 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:03.575426 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f478dfe-57bc-45c4-a1f2-8c8647719e79 service swift] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:03.589580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d40c4-105e-4e68-b69a-4326ada154aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:03.590243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d40c4-105e-4e68-b69a-4326ada154aa service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:03.590678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d40c4-105e-4e68-b69a-4326ada154aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:03.590962 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d40c4-105e-4e68-b69a-4326ada154aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:03.592859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c38d40c4-105e-4e68-b69a-4326ada154aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:03.609844 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d40c4-105e-4e68-b69a-4326ada154aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:03.609844 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d40c4-105e-4e68-b69a-4326ada154aa service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:03.621686 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1113/2229] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:05.854484 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dab6a-fd52-401b-9ba4-f1d50351f3bb None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:05.868783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dab6a-fd52-401b-9ba4-f1d50351f3bb None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:05.873678 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97280e7b-cd79-410a-ab4a-19762a3a8895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:05.875669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97280e7b-cd79-410a-ab4a-19762a3a8895 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:05.875891 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97280e7b-cd79-410a-ab4a-19762a3a8895 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:05.876197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97280e7b-cd79-410a-ab4a-19762a3a8895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:05.879767 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-97280e7b-cd79-410a-ab4a-19762a3a8895 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:05.896896 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97280e7b-cd79-410a-ab4a-19762a3a8895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:05.898790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97280e7b-cd79-410a-ab4a-19762a3a8895 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:05.912054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1117/2230] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:07.051363 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5f1a7-9d6c-4f01-aea6-159cb2d50687 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:07.051854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5f1a7-9d6c-4f01-aea6-159cb2d50687 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:07.051937 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5f1a7-9d6c-4f01-aea6-159cb2d50687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:07.078126 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-adf5f1a7-9d6c-4f01-aea6-159cb2d50687 None None] MFA Rules not processed for user `8fc940594a1848aa9644b645049a1bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:07.079425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-adf5f1a7-9d6c-4f01-aea6-159cb2d50687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:07.092273 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1114/2231] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Aug 30 14:49:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:07.136216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97280e7b-cd79-410a-ab4a-19762a3a8895 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:07.151268 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97280e7b-cd79-410a-ab4a-19762a3a8895 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:07.157726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad1a74-15c8-4f42-ac7a-6f30052290f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d98f5adfaf410580481417e7ba1638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:07.158421 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad1a74-15c8-4f42-ac7a-6f30052290f2 service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:07.158701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad1a74-15c8-4f42-ac7a-6f30052290f2 service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:07.158968 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad1a74-15c8-4f42-ac7a-6f30052290f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:07.168481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad1a74-15c8-4f42-ac7a-6f30052290f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:07.170173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad1a74-15c8-4f42-ac7a-6f30052290f2 service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:07.179756 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1118/2232] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Aug 30 14:49:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:49:15.483253 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5f1a7-9d6c-4f01-aea6-159cb2d50687 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:15.485296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-adf5f1a7-9d6c-4f01-aea6-159cb2d50687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:15.506819 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5f1a7-9d6c-4f01-aea6-159cb2d50687 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:15.513961 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b61c20-404a-4501-9d84-b15a8c2cf395 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:15.514709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b61c20-404a-4501-9d84-b15a8c2cf395 service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:15.515094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b61c20-404a-4501-9d84-b15a8c2cf395 service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:15.515425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b61c20-404a-4501-9d84-b15a8c2cf395 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:15.517643 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-52b61c20-404a-4501-9d84-b15a8c2cf395 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:15.611712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b61c20-404a-4501-9d84-b15a8c2cf395 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:15.613380 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b61c20-404a-4501-9d84-b15a8c2cf395 service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:15.632498 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1115/2233] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:15.906994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad1a74-15c8-4f42-ac7a-6f30052290f2 service swift] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:15.908723 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e3ad1a74-15c8-4f42-ac7a-6f30052290f2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:15.924835 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad1a74-15c8-4f42-ac7a-6f30052290f2 service swift] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:15.931908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79157eb4-600f-4ce0-a1cd-01fecd7e0005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:15.932966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79157eb4-600f-4ce0-a1cd-01fecd7e0005 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:15.933243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79157eb4-600f-4ce0-a1cd-01fecd7e0005 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:15.933511 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79157eb4-600f-4ce0-a1cd-01fecd7e0005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:15.935585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-79157eb4-600f-4ce0-a1cd-01fecd7e0005 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:16.032823 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79157eb4-600f-4ce0-a1cd-01fecd7e0005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:16.034240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79157eb4-600f-4ce0-a1cd-01fecd7e0005 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:16.050120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1119/2234] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:20.985508 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b61c20-404a-4501-9d84-b15a8c2cf395 service nova] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:20.996845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b61c20-404a-4501-9d84-b15a8c2cf395 service nova] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:21.001123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f171be-2570-4d0e-b382-3ec70d77ba62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:21.003261 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f171be-2570-4d0e-b382-3ec70d77ba62 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:21.003738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f171be-2570-4d0e-b382-3ec70d77ba62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:21.004183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f171be-2570-4d0e-b382-3ec70d77ba62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:21.020845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f171be-2570-4d0e-b382-3ec70d77ba62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:21.020845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f171be-2570-4d0e-b382-3ec70d77ba62 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:21.048095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1116/2235] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:24.231905 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff5c69-0d19-4566-ae2d-e075e34515af None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:24.232474 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff5c69-0d19-4566-ae2d-e075e34515af None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:24.232847 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff5c69-0d19-4566-ae2d-e075e34515af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:24.262562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f8ff5c69-0d19-4566-ae2d-e075e34515af None None] MFA Rules not processed for user `f72e1a846e864aa8b00edd8ae1a948ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:24.263274 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f8ff5c69-0d19-4566-ae2d-e075e34515af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:24.336092 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1120/2236] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:49:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:24.695677 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f171be-2570-4d0e-b382-3ec70d77ba62 service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:24.705984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f171be-2570-4d0e-b382-3ec70d77ba62 service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:24.710255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385dc12-489a-4423-9a91-4acb00027b15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:24.710943 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385dc12-489a-4423-9a91-4acb00027b15 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:24.711220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385dc12-489a-4423-9a91-4acb00027b15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:24.711597 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385dc12-489a-4423-9a91-4acb00027b15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:24.714743 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2385dc12-489a-4423-9a91-4acb00027b15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:24.731712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2385dc12-489a-4423-9a91-4acb00027b15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:24.734475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2385dc12-489a-4423-9a91-4acb00027b15 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:24.754253 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1117/2237] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:25.771870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff5c69-0d19-4566-ae2d-e075e34515af None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:25.773654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f8ff5c69-0d19-4566-ae2d-e075e34515af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:25.851648 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff5c69-0d19-4566-ae2d-e075e34515af None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:25.857744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158194fb-0681-4385-aece-7c278b96f950 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cdd6b1ee9f4e97beb9bb037e446c8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:25.858343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158194fb-0681-4385-aece-7c278b96f950 service placement] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:25.858620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158194fb-0681-4385-aece-7c278b96f950 service placement] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:25.858910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158194fb-0681-4385-aece-7c278b96f950 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:25.860674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-158194fb-0681-4385-aece-7c278b96f950 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:25.873479 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158194fb-0681-4385-aece-7c278b96f950 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:25.875596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158194fb-0681-4385-aece-7c278b96f950 service placement] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:25.888627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1121/2238] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:27.129508 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9a5be-8bff-4f5d-8b2c-5d6b1e7db6a5 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:27.131139 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9a5be-8bff-4f5d-8b2c-5d6b1e7db6a5 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:27.131912 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9a5be-8bff-4f5d-8b2c-5d6b1e7db6a5 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:27.133750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1118/2239] 10.222.0.25 () {54 vars in 921 bytes} [Tue Aug 30 14:49:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:27.146644 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7fe52-75bb-41d7-b1d9-9cf9e5569c76 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:27.148168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7fe52-75bb-41d7-b1d9-9cf9e5569c76 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:27.149438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7fe52-75bb-41d7-b1d9-9cf9e5569c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:27.171450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:27.191745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-31b7fe52-75bb-41d7-b1d9-9cf9e5569c76 None None] MFA Rules not processed for user `cc95bbdafb0c4f34ad0662bc2e3ba4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:27.193052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-31b7fe52-75bb-41d7-b1d9-9cf9e5569c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:27.204758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1122/2240] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Aug 30 14:49:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:27.224072 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9a5be-8bff-4f5d-8b2c-5d6b1e7db6a5 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:27.226484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7ac9a5be-8bff-4f5d-8b2c-5d6b1e7db6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:27.238049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=70 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:27.325671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9a5be-8bff-4f5d-8b2c-5d6b1e7db6a5 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:27.334791 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2dc719-05b4-4f4c-bf69-0375d39d8752 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69de9368993f402cb6325e5022e064d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:27.335424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dc719-05b4-4f4c-bf69-0375d39d8752 service glance] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:27.335843 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dc719-05b4-4f4c-bf69-0375d39d8752 service glance] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:27.336157 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dc719-05b4-4f4c-bf69-0375d39d8752 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:27.355633 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2dc719-05b4-4f4c-bf69-0375d39d8752 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:27.357230 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2dc719-05b4-4f4c-bf69-0375d39d8752 service glance] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:27.371030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1119/2241] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:27.465051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8901f-9b04-4a98-ab16-984cc6a1f12e None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:27.465362 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8901f-9b04-4a98-ab16-984cc6a1f12e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:27.465592 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8901f-9b04-4a98-ab16-984cc6a1f12e None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:27.467009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1123/2242] 10.222.0.25 () {54 vars in 921 bytes} [Tue Aug 30 14:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:27.474000 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b22cc-9d85-4219-900c-02601eb02d5e None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:27.474344 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b22cc-9d85-4219-900c-02601eb02d5e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:27.474602 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b22cc-9d85-4219-900c-02601eb02d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:27.496467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-830b22cc-9d85-4219-900c-02601eb02d5e None None] MFA Rules not processed for user `932fce741c5a4deba79a310a14e6bd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:27.497721 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-830b22cc-9d85-4219-900c-02601eb02d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:27.505269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1120/2243] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Aug 30 14:49:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:49:27.519359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8901f-9b04-4a98-ab16-984cc6a1f12e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:27.531879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8901f-9b04-4a98-ab16-984cc6a1f12e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:27.536322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfaa01-cf6d-4889-9b93-0dd1ade6d485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:27.536928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfaa01-cf6d-4889-9b93-0dd1ade6d485 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:27.537221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfaa01-cf6d-4889-9b93-0dd1ade6d485 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:27.537481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfaa01-cf6d-4889-9b93-0dd1ade6d485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:27.545819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfaa01-cf6d-4889-9b93-0dd1ade6d485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:27.547397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfaa01-cf6d-4889-9b93-0dd1ade6d485 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:27.556649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1124/2244] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:49:27.652864 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d26186-e463-43a0-842d-a446b4843a5f None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:27.653296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d26186-e463-43a0-842d-a446b4843a5f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:27.653686 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d26186-e463-43a0-842d-a446b4843a5f None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:27.655393 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1121/2245] 10.222.0.25 () {54 vars in 921 bytes} [Tue Aug 30 14:49:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:27.662548 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1640b65-ee48-47d0-8a3f-d5fadce1885c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:27.662948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1640b65-ee48-47d0-8a3f-d5fadce1885c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:27.663276 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1640b65-ee48-47d0-8a3f-d5fadce1885c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:27.690000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d1640b65-ee48-47d0-8a3f-d5fadce1885c None None] MFA Rules not processed for user `cc95bbdafb0c4f34ad0662bc2e3ba4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:27.691491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d1640b65-ee48-47d0-8a3f-d5fadce1885c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:27.703971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1125/2246] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Aug 30 14:49:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:49:27.716546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d26186-e463-43a0-842d-a446b4843a5f None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:27.730409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d26186-e463-43a0-842d-a446b4843a5f None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:27.734462 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f34c7-ab16-493c-940e-944762b329fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:27.735112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f34c7-ab16-493c-940e-944762b329fc service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:27.735433 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f34c7-ab16-493c-940e-944762b329fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:27.735705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f34c7-ab16-493c-940e-944762b329fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:27.748032 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f34c7-ab16-493c-940e-944762b329fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:27.749545 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f34c7-ab16-493c-940e-944762b329fc service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:27.766352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1122/2247] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:36.874358 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1640b65-ee48-47d0-8a3f-d5fadce1885c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:36.887306 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1640b65-ee48-47d0-8a3f-d5fadce1885c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:36.892040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e33f9-33c9-4122-8e77-3f8db8ed52ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:36.892746 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e33f9-33c9-4122-8e77-3f8db8ed52ff service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:36.893091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e33f9-33c9-4122-8e77-3f8db8ed52ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:36.893427 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e33f9-33c9-4122-8e77-3f8db8ed52ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:36.895447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0f1e33f9-33c9-4122-8e77-3f8db8ed52ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:36.909843 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e33f9-33c9-4122-8e77-3f8db8ed52ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:36.913607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e33f9-33c9-4122-8e77-3f8db8ed52ff service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:36.929707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1126/2248] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:37.760681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=34 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:42.967457 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8655d6-827c-4ea8-a98a-261a337dea4a None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:42.968334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8655d6-827c-4ea8-a98a-261a337dea4a None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:42.968334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8655d6-827c-4ea8-a98a-261a337dea4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:42.989771 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-fb8655d6-827c-4ea8-a98a-261a337dea4a None None] MFA Rules not processed for user `bf6b353833f14c3d819164e23374591e`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:42.991026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-fb8655d6-827c-4ea8-a98a-261a337dea4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:43.058263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1123/2249] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:49:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:43.403450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e33f9-33c9-4122-8e77-3f8db8ed52ff service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:43.413088 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e33f9-33c9-4122-8e77-3f8db8ed52ff service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:43.418066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22308d1b-5fef-4d43-8da1-a7ec07abd0a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:43.418717 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22308d1b-5fef-4d43-8da1-a7ec07abd0a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:43.418988 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22308d1b-5fef-4d43-8da1-a7ec07abd0a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:43.419253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22308d1b-5fef-4d43-8da1-a7ec07abd0a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:43.421404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-22308d1b-5fef-4d43-8da1-a7ec07abd0a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:43.433827 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22308d1b-5fef-4d43-8da1-a7ec07abd0a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:43.435643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22308d1b-5fef-4d43-8da1-a7ec07abd0a4 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:43.445486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1127/2250] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:45.336714 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178265b-28f7-4dd1-8de5-b1519d4f73d0 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:45.337316 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178265b-28f7-4dd1-8de5-b1519d4f73d0 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:45.337316 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178265b-28f7-4dd1-8de5-b1519d4f73d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:45.360354 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-0178265b-28f7-4dd1-8de5-b1519d4f73d0 None None] MFA Rules not processed for user `8fc940594a1848aa9644b645049a1bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:45.361615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0178265b-28f7-4dd1-8de5-b1519d4f73d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:45.369985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1124/2251] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Aug 30 14:49:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:45.383379 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22308d1b-5fef-4d43-8da1-a7ec07abd0a4 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:45.398500 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22308d1b-5fef-4d43-8da1-a7ec07abd0a4 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:45.406423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a7ded-beea-4af3-af36-14c9fcd85d2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d98f5adfaf410580481417e7ba1638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:45.406935 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a7ded-beea-4af3-af36-14c9fcd85d2f service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:45.407400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a7ded-beea-4af3-af36-14c9fcd85d2f service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:45.407775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a7ded-beea-4af3-af36-14c9fcd85d2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:45.418253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a7ded-beea-4af3-af36-14c9fcd85d2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:45.419901 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a7ded-beea-4af3-af36-14c9fcd85d2f service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:45.435039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1128/2252] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Aug 30 14:49:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:49:55.353944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=28 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:55.425610 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=29 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:15.029793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178265b-28f7-4dd1-8de5-b1519d4f73d0 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:15.045426 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178265b-28f7-4dd1-8de5-b1519d4f73d0 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:15.051732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b64b50-99e4-472a-a32d-c747855da8ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:15.052881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b64b50-99e4-472a-a32d-c747855da8ff service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:15.053361 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b64b50-99e4-472a-a32d-c747855da8ff service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:15.053945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b64b50-99e4-472a-a32d-c747855da8ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:15.057922 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-58b64b50-99e4-472a-a32d-c747855da8ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:15.072574 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b64b50-99e4-472a-a32d-c747855da8ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:15.074244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b64b50-99e4-472a-a32d-c747855da8ff service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:15.087022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1125/2253] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:20.645302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a7ded-beea-4af3-af36-14c9fcd85d2f service swift] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:20.658855 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a7ded-beea-4af3-af36-14c9fcd85d2f service swift] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:20.665709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de962c6-9287-40b0-ad04-a4c1f4520168 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:20.666347 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de962c6-9287-40b0-ad04-a4c1f4520168 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:20.666623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de962c6-9287-40b0-ad04-a4c1f4520168 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:20.666886 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de962c6-9287-40b0-ad04-a4c1f4520168 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:20.668909 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2de962c6-9287-40b0-ad04-a4c1f4520168 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:20.683186 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de962c6-9287-40b0-ad04-a4c1f4520168 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:20.684899 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de962c6-9287-40b0-ad04-a4c1f4520168 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:20.700366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1129/2254] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:24.485363 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27491e90-db07-4a21-a5dc-d5e24fe84587 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:24.486252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27491e90-db07-4a21-a5dc-d5e24fe84587 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:24.486414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27491e90-db07-4a21-a5dc-d5e24fe84587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:24.507946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-27491e90-db07-4a21-a5dc-d5e24fe84587 None None] MFA Rules not processed for user `8fc940594a1848aa9644b645049a1bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:24.509248 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-27491e90-db07-4a21-a5dc-d5e24fe84587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:24.516539 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1126/2255] 10.222.0.25 () {58 vars in 967 bytes} [Tue Aug 30 14:50:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:50:24.543969 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de962c6-9287-40b0-ad04-a4c1f4520168 service nova] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:24.558144 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de962c6-9287-40b0-ad04-a4c1f4520168 service nova] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:24.567881 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ec83b-b403-4039-ba7c-1fb9bcbe081f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d98f5adfaf410580481417e7ba1638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:24.569001 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ec83b-b403-4039-ba7c-1fb9bcbe081f service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:24.569647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ec83b-b403-4039-ba7c-1fb9bcbe081f service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:24.570391 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ec83b-b403-4039-ba7c-1fb9bcbe081f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:24.584563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ec83b-b403-4039-ba7c-1fb9bcbe081f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:24.587465 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ec83b-b403-4039-ba7c-1fb9bcbe081f service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:24.604110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1130/2256] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Aug 30 14:50:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:50:34.502393 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=9 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:34.596660 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=12 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:58.705546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27491e90-db07-4a21-a5dc-d5e24fe84587 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:58.719109 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27491e90-db07-4a21-a5dc-d5e24fe84587 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:58.724829 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28eeb6-2a91-46cc-9482-42fecec82a5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:58.725459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28eeb6-2a91-46cc-9482-42fecec82a5e service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:58.725738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28eeb6-2a91-46cc-9482-42fecec82a5e service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:58.726005 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28eeb6-2a91-46cc-9482-42fecec82a5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:58.727969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-ce28eeb6-2a91-46cc-9482-42fecec82a5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:58.790925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28eeb6-2a91-46cc-9482-42fecec82a5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:58.792498 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28eeb6-2a91-46cc-9482-42fecec82a5e service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:58.802391 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1127/2257] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:58 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:51:08.797985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=28 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:09.875103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ec83b-b403-4039-ba7c-1fb9bcbe081f service swift] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:51:09.892954 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ec83b-b403-4039-ba7c-1fb9bcbe081f service swift] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:51:09.906000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e7e33-b6bc-427c-a85a-c97b536e4284 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:51:09.906642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e7e33-b6bc-427c-a85a-c97b536e4284 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:09.906924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e7e33-b6bc-427c-a85a-c97b536e4284 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:09.907188 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e7e33-b6bc-427c-a85a-c97b536e4284 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:09.909227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7c2e7e33-b6bc-427c-a85a-c97b536e4284 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:10.004634 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e7e33-b6bc-427c-a85a-c97b536e4284 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:51:10.006850 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e7e33-b6bc-427c-a85a-c97b536e4284 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:51:10.023107 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1131/2258] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:51:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:51:20.015217 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=28 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:30.854139 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28eeb6-2a91-46cc-9482-42fecec82a5e service nova] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:51:30.856079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-ce28eeb6-2a91-46cc-9482-42fecec82a5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:30.872412 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28eeb6-2a91-46cc-9482-42fecec82a5e service nova] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:51:30.877623 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b57db5-5121-4ebb-a9fe-1331978f932a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cdd6b1ee9f4e97beb9bb037e446c8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:51:30.878469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b57db5-5121-4ebb-a9fe-1331978f932a service placement] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:30.878756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b57db5-5121-4ebb-a9fe-1331978f932a service placement] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:30.879009 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b57db5-5121-4ebb-a9fe-1331978f932a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:30.880969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c8b57db5-5121-4ebb-a9fe-1331978f932a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:30.894773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b57db5-5121-4ebb-a9fe-1331978f932a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:51:30.897656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b57db5-5121-4ebb-a9fe-1331978f932a service placement] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:51:30.911255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e7e33-b6bc-427c-a85a-c97b536e4284 service nova] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:51:30.913687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7c2e7e33-b6bc-427c-a85a-c97b536e4284 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:30.920207 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1128/2259] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:51:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:51:30.993575 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e7e33-b6bc-427c-a85a-c97b536e4284 service nova] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:51:30.998406 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38323de-d001-47a7-99dd-68242051a8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:51:30.999279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38323de-d001-47a7-99dd-68242051a8b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:30.999560 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38323de-d001-47a7-99dd-68242051a8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:30.999831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38323de-d001-47a7-99dd-68242051a8b3 admin admin] PATH_INFO: `/v3/users/bf6b353833f14c3d819164e23374591e` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:31.001021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38323de-d001-47a7-99dd-68242051a8b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf6b353833f14c3d819164e23374591e)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:51:31.003088 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38323de-d001-47a7-99dd-68242051a8b3 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:51:31.051885 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c38323de-d001-47a7-99dd-68242051a8b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf6b353833f14c3d819164e23374591e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:51:31.060872 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c38323de-d001-47a7-99dd-68242051a8b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf6b353833f14c3d819164e23374591e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:51:31.070470 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c38323de-d001-47a7-99dd-68242051a8b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf6b353833f14c3d819164e23374591e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:51:31.071079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c38323de-d001-47a7-99dd-68242051a8b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf6b353833f14c3d819164e23374591e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:51:31.083329 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c38323de-d001-47a7-99dd-68242051a8b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf6b353833f14c3d819164e23374591e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:51:31.089967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1132/2260] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:51:30 2022] DELETE /identity/v3/users/bf6b353833f14c3d819164e23374591e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:51:31.273198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b57db5-5121-4ebb-a9fe-1331978f932a service placement] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:51:31.284614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b57db5-5121-4ebb-a9fe-1331978f932a service placement] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:51:31.349557 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b786d-59e2-4d3a-a268-796f6eac8e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:51:31.350199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b786d-59e2-4d3a-a268-796f6eac8e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:31.350544 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b786d-59e2-4d3a-a268-796f6eac8e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:31.350797 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b786d-59e2-4d3a-a268-796f6eac8e79 admin admin] PATH_INFO: `/v3/projects/d29d875fc4d84f92801112d6d2accd13` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:31.351942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b786d-59e2-4d3a-a268-796f6eac8e79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d29d875fc4d84f92801112d6d2accd13)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:51:31.354028 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b786d-59e2-4d3a-a268-796f6eac8e79 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:51:31.404591 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cf9b786d-59e2-4d3a-a268-796f6eac8e79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd29d875fc4d84f92801112d6d2accd13', 'request_id': 'req-cf9b786d-59e2-4d3a-a268-796f6eac8e79'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:51:31.411456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cf9b786d-59e2-4d3a-a268-796f6eac8e79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd29d875fc4d84f92801112d6d2accd13', 'request_id': 'req-cf9b786d-59e2-4d3a-a268-796f6eac8e79'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:51:31.418204 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cf9b786d-59e2-4d3a-a268-796f6eac8e79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd29d875fc4d84f92801112d6d2accd13', 'request_id': 'req-cf9b786d-59e2-4d3a-a268-796f6eac8e79'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:51:31.433764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cf9b786d-59e2-4d3a-a268-796f6eac8e79 admin admin] The token cache is being invalidate because project d29d875fc4d84f92801112d6d2accd13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:51:31.434039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cf9b786d-59e2-4d3a-a268-796f6eac8e79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:51:31.434681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cf9b786d-59e2-4d3a-a268-796f6eac8e79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:51:31.436240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1129/2261] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:51:31 2022] DELETE /identity/v3/projects/d29d875fc4d84f92801112d6d2accd13 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:51:38.120308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b62b91-4783-4785-9fde-4cbce05e30d4 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:38.120937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b62b91-4783-4785-9fde-4cbce05e30d4 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:38.120937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b62b91-4783-4785-9fde-4cbce05e30d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:38.141608 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-03b62b91-4783-4785-9fde-4cbce05e30d4 None None] MFA Rules not processed for user `3398704b0dbc4af08e94b774a88851a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:51:38.142767 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-03b62b91-4783-4785-9fde-4cbce05e30d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:38.209366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1133/2262] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:51:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:51:41.430293 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:48.200476 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:52:09.214900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cceca-ea20-4dda-8f05-a6d6458db9c4 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:09.215524 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cceca-ea20-4dda-8f05-a6d6458db9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:09.215524 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cceca-ea20-4dda-8f05-a6d6458db9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:09.253047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-236cceca-ea20-4dda-8f05-a6d6458db9c4 None None] MFA Rules not processed for user `521cc5ea8fb045bc81e8e3bfa8ba239b`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:52:09.254260 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-236cceca-ea20-4dda-8f05-a6d6458db9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:09.343735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1130/2263] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:52:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:52:17.218578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b62b91-4783-4785-9fde-4cbce05e30d4 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:17.220404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-03b62b91-4783-4785-9fde-4cbce05e30d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:17.390741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b62b91-4783-4785-9fde-4cbce05e30d4 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:17.396518 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84e5d5-0807-4468-b4c4-69b1aa4e88ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:17.397122 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84e5d5-0807-4468-b4c4-69b1aa4e88ec service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:17.397391 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84e5d5-0807-4468-b4c4-69b1aa4e88ec service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:17.397656 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84e5d5-0807-4468-b4c4-69b1aa4e88ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:17.399397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9e84e5d5-0807-4468-b4c4-69b1aa4e88ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:17.466075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84e5d5-0807-4468-b4c4-69b1aa4e88ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:17.467907 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84e5d5-0807-4468-b4c4-69b1aa4e88ec service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:17.479550 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1134/2264] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:52:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:52:19.339627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=37 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:52:21.697393 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cceca-ea20-4dda-8f05-a6d6458db9c4 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:21.699744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-236cceca-ea20-4dda-8f05-a6d6458db9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:21.885428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cceca-ea20-4dda-8f05-a6d6458db9c4 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:21.890425 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd030c25-f40e-4b93-9e39-142d663f4b8a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cdd6b1ee9f4e97beb9bb037e446c8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:21.891069 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd030c25-f40e-4b93-9e39-142d663f4b8a service placement] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:21.891346 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd030c25-f40e-4b93-9e39-142d663f4b8a service placement] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:21.891734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd030c25-f40e-4b93-9e39-142d663f4b8a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:21.893953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-bd030c25-f40e-4b93-9e39-142d663f4b8a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:21.906099 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd030c25-f40e-4b93-9e39-142d663f4b8a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:21.907946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd030c25-f40e-4b93-9e39-142d663f4b8a service placement] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:21.919802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1131/2265] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:52:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:52:26.105871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84e5d5-0807-4468-b4c4-69b1aa4e88ec service nova] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:26.118743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84e5d5-0807-4468-b4c4-69b1aa4e88ec service nova] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:26.124693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae23f02-9428-4c95-8801-f0113f7187ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:26.125385 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae23f02-9428-4c95-8801-f0113f7187ed service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:26.125946 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae23f02-9428-4c95-8801-f0113f7187ed service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:26.126220 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae23f02-9428-4c95-8801-f0113f7187ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:26.128102 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1ae23f02-9428-4c95-8801-f0113f7187ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:26.141622 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae23f02-9428-4c95-8801-f0113f7187ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:26.143533 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae23f02-9428-4c95-8801-f0113f7187ed service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:26.159236 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1135/2266] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:52:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:52:31.914868 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=41 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:52:34.323468 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b6e5e-3eee-416e-a755-6d2171d9592f None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:34.324178 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b6e5e-3eee-416e-a755-6d2171d9592f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:34.324300 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b6e5e-3eee-416e-a755-6d2171d9592f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:34.345526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-549b6e5e-3eee-416e-a755-6d2171d9592f None None] MFA Rules not processed for user `f72e1a846e864aa8b00edd8ae1a948ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:52:34.347244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-549b6e5e-3eee-416e-a755-6d2171d9592f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:34.354632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1132/2267] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:52:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:52:36.155222 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=96 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:52:38.601878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae23f02-9428-4c95-8801-f0113f7187ed service nova] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:38.603539 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1ae23f02-9428-4c95-8801-f0113f7187ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:38.617789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae23f02-9428-4c95-8801-f0113f7187ed service nova] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:38.625186 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf961440-e152-4cc1-b351-b41dba46a3fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:38.625833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf961440-e152-4cc1-b351-b41dba46a3fe service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:38.626091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf961440-e152-4cc1-b351-b41dba46a3fe service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:38.626618 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf961440-e152-4cc1-b351-b41dba46a3fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:38.629323 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bf961440-e152-4cc1-b351-b41dba46a3fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:38.694859 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf961440-e152-4cc1-b351-b41dba46a3fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:38.696513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf961440-e152-4cc1-b351-b41dba46a3fe service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:38.714806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1136/2268] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:52:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:52:38.868542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54493279-b7b5-4099-986e-b192916bd465 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:38.869132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54493279-b7b5-4099-986e-b192916bd465 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:38.869132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54493279-b7b5-4099-986e-b192916bd465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:38.904744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-54493279-b7b5-4099-986e-b192916bd465 None None] MFA Rules not processed for user `06ab23cfa18e47c9871c6f451b871ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:52:38.906035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-54493279-b7b5-4099-986e-b192916bd465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:38.973605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1133/2269] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:52:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:52:48.702906 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=28 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:52:48.966356 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=28 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:52:53.179340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998b33e-ff80-424e-9409-cc6a8d469933 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:53.179871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998b33e-ff80-424e-9409-cc6a8d469933 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:53.179871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998b33e-ff80-424e-9409-cc6a8d469933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:53.202971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9998b33e-ff80-424e-9409-cc6a8d469933 None None] MFA Rules not processed for user `a1b3f34373414c61b94e2a8d9018cbd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:52:53.204289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9998b33e-ff80-424e-9409-cc6a8d469933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:53.274120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1137/2270] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:52:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:52:55.377081 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bde748-aa19-4d62-926d-b4fbc587cad5 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:55.377671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bde748-aa19-4d62-926d-b4fbc587cad5 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:55.377850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bde748-aa19-4d62-926d-b4fbc587cad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:55.410842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-56bde748-aa19-4d62-926d-b4fbc587cad5 None None] MFA Rules not processed for user `8fc940594a1848aa9644b645049a1bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:52:55.411968 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-56bde748-aa19-4d62-926d-b4fbc587cad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:55.475250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1134/2271] 10.222.0.25 () {58 vars in 967 bytes} [Tue Aug 30 14:52:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:52:55.539139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998b33e-ff80-424e-9409-cc6a8d469933 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:55.540912 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9998b33e-ff80-424e-9409-cc6a8d469933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:55.631308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998b33e-ff80-424e-9409-cc6a8d469933 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:55.636792 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11af02-8b89-4d12-8495-0a69f154dcd2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d98f5adfaf410580481417e7ba1638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:55.637630 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11af02-8b89-4d12-8495-0a69f154dcd2 service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:55.637909 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11af02-8b89-4d12-8495-0a69f154dcd2 service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:55.638175 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11af02-8b89-4d12-8495-0a69f154dcd2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:55.648663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11af02-8b89-4d12-8495-0a69f154dcd2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:55.650057 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11af02-8b89-4d12-8495-0a69f154dcd2 service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:55.660317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1138/2272] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Aug 30 14:52:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:52:56.093974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bde748-aa19-4d62-926d-b4fbc587cad5 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:56.096399 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-56bde748-aa19-4d62-926d-b4fbc587cad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:56.191119 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bde748-aa19-4d62-926d-b4fbc587cad5 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:56.194877 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08064c9-f6f9-4e3a-b176-eef01d95a976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:56.195497 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08064c9-f6f9-4e3a-b176-eef01d95a976 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:56.195775 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08064c9-f6f9-4e3a-b176-eef01d95a976 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:56.196068 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08064c9-f6f9-4e3a-b176-eef01d95a976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:56.197890 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e08064c9-f6f9-4e3a-b176-eef01d95a976 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:56.295239 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08064c9-f6f9-4e3a-b176-eef01d95a976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:56.296739 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08064c9-f6f9-4e3a-b176-eef01d95a976 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:56.313374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:52:56.318962 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1135/2273] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:53:00.620327 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11af02-8b89-4d12-8495-0a69f154dcd2 service swift] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:00.633378 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11af02-8b89-4d12-8495-0a69f154dcd2 service swift] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:00.639498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89622d0-9bfb-42bc-a7c2-be2c4c0aa671 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:00.640163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89622d0-9bfb-42bc-a7c2-be2c4c0aa671 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:00.640443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89622d0-9bfb-42bc-a7c2-be2c4c0aa671 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:00.640713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89622d0-9bfb-42bc-a7c2-be2c4c0aa671 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:00.642695 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c89622d0-9bfb-42bc-a7c2-be2c4c0aa671 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:00.655762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89622d0-9bfb-42bc-a7c2-be2c4c0aa671 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:00.657555 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89622d0-9bfb-42bc-a7c2-be2c4c0aa671 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:00.669174 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1139/2274] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:53:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:53:06.317950 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:07.218870 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2316d7-0bb0-4998-be68-ea2e8f1894b7 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:07.219570 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2316d7-0bb0-4998-be68-ea2e8f1894b7 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:07.220116 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2316d7-0bb0-4998-be68-ea2e8f1894b7 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:07.222163 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1136/2275] 10.222.0.25 () {54 vars in 921 bytes} [Tue Aug 30 14:53:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:53:07.244035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba9576-bf9f-439c-8480-8933e6297675 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:07.245001 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba9576-bf9f-439c-8480-8933e6297675 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:07.245430 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba9576-bf9f-439c-8480-8933e6297675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:07.307860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e7ba9576-bf9f-439c-8480-8933e6297675 None None] MFA Rules not processed for user `d7cdd6b1ee9f4e97beb9bb037e446c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:07.309621 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e7ba9576-bf9f-439c-8480-8933e6297675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:07.324736 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1140/2276] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Aug 30 14:53:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:53:07.354063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2316d7-0bb0-4998-be68-ea2e8f1894b7 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:07.358746 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0b2316d7-0bb0-4998-be68-ea2e8f1894b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:07.363070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba9576-bf9f-439c-8480-8933e6297675 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:07.376081 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba9576-bf9f-439c-8480-8933e6297675 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:07.384063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74654fb3-f90d-434e-a702-514c8fbe4437 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cdd6b1ee9f4e97beb9bb037e446c8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:07.384708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74654fb3-f90d-434e-a702-514c8fbe4437 service placement] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:07.385542 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74654fb3-f90d-434e-a702-514c8fbe4437 service placement] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:07.385820 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74654fb3-f90d-434e-a702-514c8fbe4437 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:07.387089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2316d7-0bb0-4998-be68-ea2e8f1894b7 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:07.393255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5dfb3-36de-473d-880c-d12a64681df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:07.393936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5dfb3-36de-473d-880c-d12a64681df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:07.394218 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5dfb3-36de-473d-880c-d12a64681df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:07.394487 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5dfb3-36de-473d-880c-d12a64681df3 admin admin] PATH_INFO: `/v3/users/eb1aa3917d3d427e91f0805e5d6ab4eb` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:07.396026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5dfb3-36de-473d-880c-d12a64681df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb1aa3917d3d427e91f0805e5d6ab4eb)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:07.397857 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74654fb3-f90d-434e-a702-514c8fbe4437 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:07.398559 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5dfb3-36de-473d-880c-d12a64681df3 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:07.399535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74654fb3-f90d-434e-a702-514c8fbe4437 service placement] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:07.411780 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1141/2277] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:53:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:53:07.486775 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-c6a5dfb3-36de-473d-880c-d12a64681df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb1aa3917d3d427e91f0805e5d6ab4eb'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:07.487925 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-c6a5dfb3-36de-473d-880c-d12a64681df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb1aa3917d3d427e91f0805e5d6ab4eb'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:07.490310 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-c6a5dfb3-36de-473d-880c-d12a64681df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb1aa3917d3d427e91f0805e5d6ab4eb'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:07.500983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-c6a5dfb3-36de-473d-880c-d12a64681df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb1aa3917d3d427e91f0805e5d6ab4eb'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:07.521473 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-c6a5dfb3-36de-473d-880c-d12a64681df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb1aa3917d3d427e91f0805e5d6ab4eb'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:07.537122 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1137/2278] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:53:07 2022] DELETE /identity/v3/users/eb1aa3917d3d427e91f0805e5d6ab4eb => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:07.741971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74654fb3-f90d-434e-a702-514c8fbe4437 service placement] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:07.759581 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74654fb3-f90d-434e-a702-514c8fbe4437 service placement] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:07.813137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:07.833307 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8737f-cd74-410f-b4bc-3c275a865cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:07.833949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8737f-cd74-410f-b4bc-3c275a865cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:07.834346 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8737f-cd74-410f-b4bc-3c275a865cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:07.834616 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8737f-cd74-410f-b4bc-3c275a865cdd admin admin] PATH_INFO: `/v3/projects/30338c95ad8d476b89bbec65ad188b2d` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:07.835735 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8737f-cd74-410f-b4bc-3c275a865cdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=30338c95ad8d476b89bbec65ad188b2d)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:07.837717 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8737f-cd74-410f-b4bc-3c275a865cdd admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:07.894831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bbf8737f-cd74-410f-b4bc-3c275a865cdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30338c95ad8d476b89bbec65ad188b2d', 'request_id': 'req-bbf8737f-cd74-410f-b4bc-3c275a865cdd'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:07.904278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bbf8737f-cd74-410f-b4bc-3c275a865cdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30338c95ad8d476b89bbec65ad188b2d', 'request_id': 'req-bbf8737f-cd74-410f-b4bc-3c275a865cdd'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:07.911455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bbf8737f-cd74-410f-b4bc-3c275a865cdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30338c95ad8d476b89bbec65ad188b2d', 'request_id': 'req-bbf8737f-cd74-410f-b4bc-3c275a865cdd'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:07.927043 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bbf8737f-cd74-410f-b4bc-3c275a865cdd admin admin] The token cache is being invalidate because project 30338c95ad8d476b89bbec65ad188b2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:53:07.927328 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bbf8737f-cd74-410f-b4bc-3c275a865cdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:07.928214 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bbf8737f-cd74-410f-b4bc-3c275a865cdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:07.930141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1142/2279] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:53:07 2022] DELETE /identity/v3/projects/30338c95ad8d476b89bbec65ad188b2d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:07.934823 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5dfb3-36de-473d-880c-d12a64681df3 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:07.936902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c6a5dfb3-36de-473d-880c-d12a64681df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:07.952172 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5dfb3-36de-473d-880c-d12a64681df3 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:08.025545 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799cff3-79e0-4704-99e8-2ea56b1039f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:08.026299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799cff3-79e0-4704-99e8-2ea56b1039f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:08.026637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799cff3-79e0-4704-99e8-2ea56b1039f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:08.027015 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799cff3-79e0-4704-99e8-2ea56b1039f5 admin admin] PATH_INFO: `/v3/users/aefb376ac5874039bce826475943fe15` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:08.028224 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799cff3-79e0-4704-99e8-2ea56b1039f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aefb376ac5874039bce826475943fe15)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:08.030244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799cff3-79e0-4704-99e8-2ea56b1039f5 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:08.083257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-0799cff3-79e0-4704-99e8-2ea56b1039f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aefb376ac5874039bce826475943fe15'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:08.083944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-0799cff3-79e0-4704-99e8-2ea56b1039f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aefb376ac5874039bce826475943fe15'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:08.084705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-0799cff3-79e0-4704-99e8-2ea56b1039f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aefb376ac5874039bce826475943fe15'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:08.102583 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-0799cff3-79e0-4704-99e8-2ea56b1039f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aefb376ac5874039bce826475943fe15'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:08.115285 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-0799cff3-79e0-4704-99e8-2ea56b1039f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aefb376ac5874039bce826475943fe15'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:08.117764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:08.131456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1138/2280] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:53:07 2022] DELETE /identity/v3/users/aefb376ac5874039bce826475943fe15 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:08.329855 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8737f-cd74-410f-b4bc-3c275a865cdd admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:08.332347 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bbf8737f-cd74-410f-b4bc-3c275a865cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:08.351551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8737f-cd74-410f-b4bc-3c275a865cdd admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:08.416604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce58bfe-b74d-43d9-b059-b115c2d04a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:08.417459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce58bfe-b74d-43d9-b059-b115c2d04a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:08.417752 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce58bfe-b74d-43d9-b059-b115c2d04a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:08.418158 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce58bfe-b74d-43d9-b059-b115c2d04a31 admin admin] PATH_INFO: `/v3/projects/9df09fbfd5084868a818e53fe4550329` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:08.419166 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce58bfe-b74d-43d9-b059-b115c2d04a31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9df09fbfd5084868a818e53fe4550329)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:08.421308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce58bfe-b74d-43d9-b059-b115c2d04a31 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:08.498687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9ce58bfe-b74d-43d9-b059-b115c2d04a31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9df09fbfd5084868a818e53fe4550329', 'request_id': 'req-9ce58bfe-b74d-43d9-b059-b115c2d04a31'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:08.508284 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9ce58bfe-b74d-43d9-b059-b115c2d04a31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9df09fbfd5084868a818e53fe4550329', 'request_id': 'req-9ce58bfe-b74d-43d9-b059-b115c2d04a31'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:08.515991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9ce58bfe-b74d-43d9-b059-b115c2d04a31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9df09fbfd5084868a818e53fe4550329', 'request_id': 'req-9ce58bfe-b74d-43d9-b059-b115c2d04a31'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:08.526255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:08.530829 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9ce58bfe-b74d-43d9-b059-b115c2d04a31 admin admin] The token cache is being invalidate because project 9df09fbfd5084868a818e53fe4550329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:53:08.531938 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9ce58bfe-b74d-43d9-b059-b115c2d04a31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:08.533392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9ce58bfe-b74d-43d9-b059-b115c2d04a31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:08.552082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1143/2281] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:53:08 2022] DELETE /identity/v3/projects/9df09fbfd5084868a818e53fe4550329 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:18.001808 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799cff3-79e0-4704-99e8-2ea56b1039f5 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:18.003264 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0799cff3-79e0-4704-99e8-2ea56b1039f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:18.061416 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799cff3-79e0-4704-99e8-2ea56b1039f5 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:18.125249 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8a46b-30ce-4d28-9ef4-8390bf3baa2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:18.125870 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8a46b-30ce-4d28-9ef4-8390bf3baa2e service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:18.126144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8a46b-30ce-4d28-9ef4-8390bf3baa2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:18.126412 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8a46b-30ce-4d28-9ef4-8390bf3baa2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:18.128264 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-51f8a46b-30ce-4d28-9ef4-8390bf3baa2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:18.195895 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8a46b-30ce-4d28-9ef4-8390bf3baa2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:18.197619 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8a46b-30ce-4d28-9ef4-8390bf3baa2e service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:18.208532 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1139/2282] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:53:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:53:21.296895 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce58bfe-b74d-43d9-b059-b115c2d04a31 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:21.298350 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9ce58bfe-b74d-43d9-b059-b115c2d04a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:21.311682 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce58bfe-b74d-43d9-b059-b115c2d04a31 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:21.316663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89202cf-a719-4082-abe8-1274060c9198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:21.317301 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89202cf-a719-4082-abe8-1274060c9198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:21.317570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89202cf-a719-4082-abe8-1274060c9198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:21.317842 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89202cf-a719-4082-abe8-1274060c9198 admin admin] PATH_INFO: `/v3/users/3398704b0dbc4af08e94b774a88851a4` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:21.319060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89202cf-a719-4082-abe8-1274060c9198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3398704b0dbc4af08e94b774a88851a4)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:21.321080 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89202cf-a719-4082-abe8-1274060c9198 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:21.371610 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f89202cf-a719-4082-abe8-1274060c9198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3398704b0dbc4af08e94b774a88851a4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:21.378622 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f89202cf-a719-4082-abe8-1274060c9198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3398704b0dbc4af08e94b774a88851a4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:21.387400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f89202cf-a719-4082-abe8-1274060c9198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3398704b0dbc4af08e94b774a88851a4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:21.388118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f89202cf-a719-4082-abe8-1274060c9198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3398704b0dbc4af08e94b774a88851a4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:21.401194 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f89202cf-a719-4082-abe8-1274060c9198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3398704b0dbc4af08e94b774a88851a4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:21.407903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1144/2283] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:53:21 2022] DELETE /identity/v3/users/3398704b0dbc4af08e94b774a88851a4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:21.603481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8a46b-30ce-4d28-9ef4-8390bf3baa2e service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:21.621285 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8a46b-30ce-4d28-9ef4-8390bf3baa2e service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:21.683980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:21.690455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f70da-621c-4cbd-8e29-10b469c770a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:21.691096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f70da-621c-4cbd-8e29-10b469c770a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:21.691373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f70da-621c-4cbd-8e29-10b469c770a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:21.691644 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f70da-621c-4cbd-8e29-10b469c770a3 admin admin] PATH_INFO: `/v3/projects/52028232fa7e4d808057e3e5512e6135` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:21.692791 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f70da-621c-4cbd-8e29-10b469c770a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52028232fa7e4d808057e3e5512e6135)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:21.694773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f70da-621c-4cbd-8e29-10b469c770a3 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:21.744808 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b64f70da-621c-4cbd-8e29-10b469c770a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52028232fa7e4d808057e3e5512e6135', 'request_id': 'req-b64f70da-621c-4cbd-8e29-10b469c770a3'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:21.751222 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b64f70da-621c-4cbd-8e29-10b469c770a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52028232fa7e4d808057e3e5512e6135', 'request_id': 'req-b64f70da-621c-4cbd-8e29-10b469c770a3'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:21.758126 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b64f70da-621c-4cbd-8e29-10b469c770a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52028232fa7e4d808057e3e5512e6135', 'request_id': 'req-b64f70da-621c-4cbd-8e29-10b469c770a3'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:21.776934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b64f70da-621c-4cbd-8e29-10b469c770a3 admin admin] The token cache is being invalidate because project 52028232fa7e4d808057e3e5512e6135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:53:21.777227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b64f70da-621c-4cbd-8e29-10b469c770a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:21.777943 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b64f70da-621c-4cbd-8e29-10b469c770a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:21.779617 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1140/2284] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:53:21 2022] DELETE /identity/v3/projects/52028232fa7e4d808057e3e5512e6135 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:53:21.785035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b7d12-7b7a-45ae-b57a-d44f87f65748 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:21.785287 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b7d12-7b7a-45ae-b57a-d44f87f65748 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:21.785590 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b7d12-7b7a-45ae-b57a-d44f87f65748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:21.846162 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ec3b7d12-7b7a-45ae-b57a-d44f87f65748 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:21.847411 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ec3b7d12-7b7a-45ae-b57a-d44f87f65748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:21.922780 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1145/2285] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:21.933426 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334cd36-14a5-4bd2-aa49-97e966185535 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:21.933698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334cd36-14a5-4bd2-aa49-97e966185535 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:21.934009 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334cd36-14a5-4bd2-aa49-97e966185535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:21.959523 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-6334cd36-14a5-4bd2-aa49-97e966185535 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:21.960889 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6334cd36-14a5-4bd2-aa49-97e966185535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:21.970413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1141/2286] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:21.975567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b7d12-7b7a-45ae-b57a-d44f87f65748 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:21.986037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b7d12-7b7a-45ae-b57a-d44f87f65748 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:21.991886 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a2aa7-a8ba-4ea8-8093-31189fb2ef66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:21.992545 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a2aa7-a8ba-4ea8-8093-31189fb2ef66 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:21.992819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a2aa7-a8ba-4ea8-8093-31189fb2ef66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:21.993084 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a2aa7-a8ba-4ea8-8093-31189fb2ef66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:21.993741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a2aa7-a8ba-4ea8-8093-31189fb2ef66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:53:21.994042 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a2aa7-a8ba-4ea8-8093-31189fb2ef66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:21.995756 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a2aa7-a8ba-4ea8-8093-31189fb2ef66 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:22.008800 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1146/2287] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:53:22.013415 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334cd36-14a5-4bd2-aa49-97e966185535 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:22.023011 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334cd36-14a5-4bd2-aa49-97e966185535 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:22.028632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f9dc9-4a9b-4dac-a7d5-50e3e80fd423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:22.029223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f9dc9-4a9b-4dac-a7d5-50e3e80fd423 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:22.029512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f9dc9-4a9b-4dac-a7d5-50e3e80fd423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:22.029805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f9dc9-4a9b-4dac-a7d5-50e3e80fd423 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:22.030436 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f9dc9-4a9b-4dac-a7d5-50e3e80fd423 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:22.032397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f9dc9-4a9b-4dac-a7d5-50e3e80fd423 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:22.063942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1142/2288] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:53:22 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:53:22.068291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a2aa7-a8ba-4ea8-8093-31189fb2ef66 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:22.079489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a2aa7-a8ba-4ea8-8093-31189fb2ef66 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:22.113510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:22.139525 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a813c-657d-45ed-adfe-fc8f533ba774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:22.140200 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a813c-657d-45ed-adfe-fc8f533ba774 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:22.140520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a813c-657d-45ed-adfe-fc8f533ba774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:22.140827 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a813c-657d-45ed-adfe-fc8f533ba774 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:22.141461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a813c-657d-45ed-adfe-fc8f533ba774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:22.143370 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a813c-657d-45ed-adfe-fc8f533ba774 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:22.167865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1147/2289] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:53:22 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:53:22.172468 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f9dc9-4a9b-4dac-a7d5-50e3e80fd423 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:22.187716 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f9dc9-4a9b-4dac-a7d5-50e3e80fd423 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:22.193312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e765e-e384-4746-ad60-ec28d5b9681e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:22.193933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e765e-e384-4746-ad60-ec28d5b9681e admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:22.194388 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e765e-e384-4746-ad60-ec28d5b9681e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:22.194718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e765e-e384-4746-ad60-ec28d5b9681e admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:22.195560 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e765e-e384-4746-ad60-ec28d5b9681e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:22.197437 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e765e-e384-4746-ad60-ec28d5b9681e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:22.212224 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1143/2290] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:53:22.214513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a813c-657d-45ed-adfe-fc8f533ba774 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:22.226143 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a813c-657d-45ed-adfe-fc8f533ba774 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:22.232734 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0ee20-a38d-40ad-87d6-c2b2fd99497f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:22.233807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0ee20-a38d-40ad-87d6-c2b2fd99497f admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:22.234187 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0ee20-a38d-40ad-87d6-c2b2fd99497f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:22.234621 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0ee20-a38d-40ad-87d6-c2b2fd99497f admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:22.235702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0ee20-a38d-40ad-87d6-c2b2fd99497f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:22.238217 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0ee20-a38d-40ad-87d6-c2b2fd99497f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:22.254579 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1148/2291] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:53:22.261302 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e765e-e384-4746-ad60-ec28d5b9681e admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:22.273130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e765e-e384-4746-ad60-ec28d5b9681e admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:22.278953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d0f60-d0f0-4056-b9c7-c3ff64574733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:22.279600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d0f60-d0f0-4056-b9c7-c3ff64574733 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:22.279874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d0f60-d0f0-4056-b9c7-c3ff64574733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:22.280171 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d0f60-d0f0-4056-b9c7-c3ff64574733 admin admin] PATH_INFO: `/v3/projects/a77980816b194115939548358ebfccc0/users/bfdc7f26a155404e9c46bb70685d911d/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:22.296273 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d0f60-d0f0-4056-b9c7-c3ff64574733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a77980816b194115939548358ebfccc0, user_id=bfdc7f26a155404e9c46bb70685d911d, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:22.298912 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d0f60-d0f0-4056-b9c7-c3ff64574733 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:22.316691 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1144/2292] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:53:22 2022] PUT /identity/v3/projects/a77980816b194115939548358ebfccc0/users/bfdc7f26a155404e9c46bb70685d911d/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:53:22.327759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce732ae-6e9b-4b57-93d9-fe186b9a54de None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:22.328289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce732ae-6e9b-4b57-93d9-fe186b9a54de None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:22.328289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce732ae-6e9b-4b57-93d9-fe186b9a54de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:22.359970 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4ce732ae-6e9b-4b57-93d9-fe186b9a54de None None] MFA Rules not processed for user `bfdc7f26a155404e9c46bb70685d911d`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:22.361217 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4ce732ae-6e9b-4b57-93d9-fe186b9a54de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:22.431965 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1149/2293] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:22.436744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d0f60-d0f0-4056-b9c7-c3ff64574733 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:22.507772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d0f60-d0f0-4056-b9c7-c3ff64574733 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:22.513701 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c57d5a-ffb1-40ae-9688-191c5952dd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:22.514346 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c57d5a-ffb1-40ae-9688-191c5952dd22 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:22.514619 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c57d5a-ffb1-40ae-9688-191c5952dd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:22.514883 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c57d5a-ffb1-40ae-9688-191c5952dd22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:22.515530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c57d5a-ffb1-40ae-9688-191c5952dd22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:22.517597 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c57d5a-ffb1-40ae-9688-191c5952dd22 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:22.547383 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1145/2294] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:53:22 2022] POST /identity/v3/projects => generated 363 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:53:22.551597 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce732ae-6e9b-4b57-93d9-fe186b9a54de None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:22.563473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce732ae-6e9b-4b57-93d9-fe186b9a54de None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:22.628411 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfaa5b9-06c2-48a6-99d1-d1cb75b0a145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:22.629069 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaa5b9-06c2-48a6-99d1-d1cb75b0a145 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:22.629367 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaa5b9-06c2-48a6-99d1-d1cb75b0a145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:22.629624 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaa5b9-06c2-48a6-99d1-d1cb75b0a145 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:22.630254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfaa5b9-06c2-48a6-99d1-d1cb75b0a145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:22.632283 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfaa5b9-06c2-48a6-99d1-d1cb75b0a145 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:22.658570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1150/2295] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:53:22 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:53:22.664235 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c57d5a-ffb1-40ae-9688-191c5952dd22 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:22.675689 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c57d5a-ffb1-40ae-9688-191c5952dd22 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:22.682148 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cba6c2-327c-444f-8ab0-44ef59599eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:22.682757 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cba6c2-327c-444f-8ab0-44ef59599eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:22.683037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cba6c2-327c-444f-8ab0-44ef59599eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:22.683311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cba6c2-327c-444f-8ab0-44ef59599eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:22.683964 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cba6c2-327c-444f-8ab0-44ef59599eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:22.685629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cba6c2-327c-444f-8ab0-44ef59599eb9 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:22.687831 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:22.701033 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1146/2296] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:53:22.702316 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfaa5b9-06c2-48a6-99d1-d1cb75b0a145 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:22.712842 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfaa5b9-06c2-48a6-99d1-d1cb75b0a145 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:22.718544 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd596e3-3d40-447b-8d9d-da147033ea86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:22.719198 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd596e3-3d40-447b-8d9d-da147033ea86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:22.719476 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd596e3-3d40-447b-8d9d-da147033ea86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:22.719746 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd596e3-3d40-447b-8d9d-da147033ea86 admin admin] PATH_INFO: `/v3/projects/25303813bbee45b1becc97f9e9a68e88/users/cbe6d4659f9b490ab3578112c873bb12/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:22.737494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd596e3-3d40-447b-8d9d-da147033ea86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25303813bbee45b1becc97f9e9a68e88, user_id=cbe6d4659f9b490ab3578112c873bb12, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:22.740547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd596e3-3d40-447b-8d9d-da147033ea86 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:22.758519 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1151/2297] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:53:22 2022] PUT /identity/v3/projects/25303813bbee45b1becc97f9e9a68e88/users/cbe6d4659f9b490ab3578112c873bb12/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:22.762602 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cba6c2-327c-444f-8ab0-44ef59599eb9 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:22.773254 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cba6c2-327c-444f-8ab0-44ef59599eb9 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:22.841112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6509c-833c-4789-9ce5-a402f82e3c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:22.841859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6509c-833c-4789-9ce5-a402f82e3c48 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:22.842224 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6509c-833c-4789-9ce5-a402f82e3c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:22.842518 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6509c-833c-4789-9ce5-a402f82e3c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:22.843595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6509c-833c-4789-9ce5-a402f82e3c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:22.846160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6509c-833c-4789-9ce5-a402f82e3c48 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:22.857991 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1147/2298] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:53:22.862483 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd596e3-3d40-447b-8d9d-da147033ea86 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:22.874294 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd596e3-3d40-447b-8d9d-da147033ea86 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:22.880276 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ec1fc-c294-4f58-a117-9aae2d49fe75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:22.880932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ec1fc-c294-4f58-a117-9aae2d49fe75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:22.881211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ec1fc-c294-4f58-a117-9aae2d49fe75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:22.881483 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ec1fc-c294-4f58-a117-9aae2d49fe75 admin admin] PATH_INFO: `/v3/projects/25303813bbee45b1becc97f9e9a68e88/users/cbe6d4659f9b490ab3578112c873bb12/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:22.883684 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ec1fc-c294-4f58-a117-9aae2d49fe75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25303813bbee45b1becc97f9e9a68e88, user_id=cbe6d4659f9b490ab3578112c873bb12, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:22.886372 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ec1fc-c294-4f58-a117-9aae2d49fe75 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:22.892833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:22.902700 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1152/2299] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:53:22 2022] PUT /identity/v3/projects/25303813bbee45b1becc97f9e9a68e88/users/cbe6d4659f9b490ab3578112c873bb12/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:22.917554 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2591c4-bee5-42bd-8a8c-2aba40e091bc None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:22.917906 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2591c4-bee5-42bd-8a8c-2aba40e091bc None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:22.918157 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2591c4-bee5-42bd-8a8c-2aba40e091bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:22.949895 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-9c2591c4-bee5-42bd-8a8c-2aba40e091bc None None] MFA Rules not processed for user `cbe6d4659f9b490ab3578112c873bb12`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:22.951134 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-9c2591c4-bee5-42bd-8a8c-2aba40e091bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:23.038569 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1148/2300] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:23.053348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ec1fc-c294-4f58-a117-9aae2d49fe75 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:23.055452 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-272ec1fc-c294-4f58-a117-9aae2d49fe75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:23.138392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ec1fc-c294-4f58-a117-9aae2d49fe75 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:23.142528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6955fc-013b-4f51-832a-76fa45f8218d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:23.143137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6955fc-013b-4f51-832a-76fa45f8218d service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:23.143405 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6955fc-013b-4f51-832a-76fa45f8218d service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:23.143668 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6955fc-013b-4f51-832a-76fa45f8218d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:23.153529 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6955fc-013b-4f51-832a-76fa45f8218d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:23.155003 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6955fc-013b-4f51-832a-76fa45f8218d service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:23.164529 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1153/2301] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:53:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:53:23.629006 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2591c4-bee5-42bd-8a8c-2aba40e091bc None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:23.695043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2591c4-bee5-42bd-8a8c-2aba40e091bc None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:23.700124 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731ea4d-2379-400c-8018-08589a571476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:23.700783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731ea4d-2379-400c-8018-08589a571476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:23.701085 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731ea4d-2379-400c-8018-08589a571476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:23.701352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731ea4d-2379-400c-8018-08589a571476 admin admin] PATH_INFO: `/v3/users/bfdc7f26a155404e9c46bb70685d911d` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:23.702667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731ea4d-2379-400c-8018-08589a571476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfdc7f26a155404e9c46bb70685d911d)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:23.704640 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731ea4d-2379-400c-8018-08589a571476 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:23.733486 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:23.762846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5731ea4d-2379-400c-8018-08589a571476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfdc7f26a155404e9c46bb70685d911d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:23.763522 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5731ea4d-2379-400c-8018-08589a571476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfdc7f26a155404e9c46bb70685d911d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:23.764189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5731ea4d-2379-400c-8018-08589a571476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfdc7f26a155404e9c46bb70685d911d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:23.773158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5731ea4d-2379-400c-8018-08589a571476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfdc7f26a155404e9c46bb70685d911d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:23.787696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5731ea4d-2379-400c-8018-08589a571476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfdc7f26a155404e9c46bb70685d911d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:23.800104 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1149/2302] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:53:23 2022] DELETE /identity/v3/users/bfdc7f26a155404e9c46bb70685d911d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:23.805257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079607d4-6c24-486b-b8b7-5a41e033e038 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:23.805745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079607d4-6c24-486b-b8b7-5a41e033e038 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:23.805745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079607d4-6c24-486b-b8b7-5a41e033e038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:23.826103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-079607d4-6c24-486b-b8b7-5a41e033e038 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:23.827507 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-079607d4-6c24-486b-b8b7-5a41e033e038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:23.890997 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1154/2303] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:23.906015 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731ea4d-2379-400c-8018-08589a571476 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:23.907965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5731ea4d-2379-400c-8018-08589a571476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:23.974363 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731ea4d-2379-400c-8018-08589a571476 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:23.978784 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9600e18-8164-4aa8-a9ad-53bae32f62ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:23.979675 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9600e18-8164-4aa8-a9ad-53bae32f62ed service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:23.979959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9600e18-8164-4aa8-a9ad-53bae32f62ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:23.980265 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9600e18-8164-4aa8-a9ad-53bae32f62ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:23.990101 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9600e18-8164-4aa8-a9ad-53bae32f62ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:23.991539 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9600e18-8164-4aa8-a9ad-53bae32f62ed service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:24.002386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1150/2304] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:53:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:53:24.291713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079607d4-6c24-486b-b8b7-5a41e033e038 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:24.293378 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-079607d4-6c24-486b-b8b7-5a41e033e038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:24.312433 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079607d4-6c24-486b-b8b7-5a41e033e038 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:24.317540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4b593-81b2-42ff-8e46-44d6eae30149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:24.318126 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4b593-81b2-42ff-8e46-44d6eae30149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:24.318414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4b593-81b2-42ff-8e46-44d6eae30149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:24.318654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4b593-81b2-42ff-8e46-44d6eae30149 admin admin] PATH_INFO: `/v3/projects/a77980816b194115939548358ebfccc0` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:24.320634 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc4b593-81b2-42ff-8e46-44d6eae30149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a77980816b194115939548358ebfccc0)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:24.322694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc4b593-81b2-42ff-8e46-44d6eae30149 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:24.370938 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7cc4b593-81b2-42ff-8e46-44d6eae30149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a77980816b194115939548358ebfccc0', 'request_id': 'req-7cc4b593-81b2-42ff-8e46-44d6eae30149'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:24.380298 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7cc4b593-81b2-42ff-8e46-44d6eae30149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a77980816b194115939548358ebfccc0', 'request_id': 'req-7cc4b593-81b2-42ff-8e46-44d6eae30149'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:24.386308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7cc4b593-81b2-42ff-8e46-44d6eae30149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a77980816b194115939548358ebfccc0', 'request_id': 'req-7cc4b593-81b2-42ff-8e46-44d6eae30149'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:24.397218 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7cc4b593-81b2-42ff-8e46-44d6eae30149 admin admin] The token cache is being invalidate because project a77980816b194115939548358ebfccc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:53:24.397499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7cc4b593-81b2-42ff-8e46-44d6eae30149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:24.398166 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7cc4b593-81b2-42ff-8e46-44d6eae30149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:24.399706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1155/2305] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:53:24 2022] DELETE /identity/v3/projects/a77980816b194115939548358ebfccc0 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:24.404481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9600e18-8164-4aa8-a9ad-53bae32f62ed service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:24.415191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9600e18-8164-4aa8-a9ad-53bae32f62ed service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:24.477618 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7dcb3-eeaa-46ed-b268-c99facff76c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:24.478323 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7dcb3-eeaa-46ed-b268-c99facff76c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:24.478603 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7dcb3-eeaa-46ed-b268-c99facff76c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:24.478884 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7dcb3-eeaa-46ed-b268-c99facff76c5 admin admin] PATH_INFO: `/v3/users/cbe6d4659f9b490ab3578112c873bb12` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:24.480150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7dcb3-eeaa-46ed-b268-c99facff76c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbe6d4659f9b490ab3578112c873bb12)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:24.482228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7dcb3-eeaa-46ed-b268-c99facff76c5 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:24.532423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:24.534844 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8ac7dcb3-eeaa-46ed-b268-c99facff76c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbe6d4659f9b490ab3578112c873bb12'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:24.535583 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8ac7dcb3-eeaa-46ed-b268-c99facff76c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbe6d4659f9b490ab3578112c873bb12'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:24.536465 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8ac7dcb3-eeaa-46ed-b268-c99facff76c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbe6d4659f9b490ab3578112c873bb12'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:24.546965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8ac7dcb3-eeaa-46ed-b268-c99facff76c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbe6d4659f9b490ab3578112c873bb12'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:24.560395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8ac7dcb3-eeaa-46ed-b268-c99facff76c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbe6d4659f9b490ab3578112c873bb12'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:24.573528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1151/2306] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:53:24 2022] DELETE /identity/v3/users/cbe6d4659f9b490ab3578112c873bb12 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:24.863089 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4b593-81b2-42ff-8e46-44d6eae30149 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:24.864709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7cc4b593-81b2-42ff-8e46-44d6eae30149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:24.877331 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:24.882044 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4b593-81b2-42ff-8e46-44d6eae30149 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:24.947205 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07400a93-5cc3-4adf-b3a8-655c703b2d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:24.948056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07400a93-5cc3-4adf-b3a8-655c703b2d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:24.948298 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07400a93-5cc3-4adf-b3a8-655c703b2d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:24.948641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07400a93-5cc3-4adf-b3a8-655c703b2d94 admin admin] PATH_INFO: `/v3/projects/25303813bbee45b1becc97f9e9a68e88` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:24.949885 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07400a93-5cc3-4adf-b3a8-655c703b2d94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25303813bbee45b1becc97f9e9a68e88)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:24.951828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07400a93-5cc3-4adf-b3a8-655c703b2d94 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:25.003755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-07400a93-5cc3-4adf-b3a8-655c703b2d94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25303813bbee45b1becc97f9e9a68e88', 'request_id': 'req-07400a93-5cc3-4adf-b3a8-655c703b2d94'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:25.014551 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-07400a93-5cc3-4adf-b3a8-655c703b2d94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25303813bbee45b1becc97f9e9a68e88', 'request_id': 'req-07400a93-5cc3-4adf-b3a8-655c703b2d94'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:25.023286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-07400a93-5cc3-4adf-b3a8-655c703b2d94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25303813bbee45b1becc97f9e9a68e88', 'request_id': 'req-07400a93-5cc3-4adf-b3a8-655c703b2d94'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:25.041239 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-07400a93-5cc3-4adf-b3a8-655c703b2d94 admin admin] The token cache is being invalidate because project 25303813bbee45b1becc97f9e9a68e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:53:25.041530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-07400a93-5cc3-4adf-b3a8-655c703b2d94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:25.042292 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-07400a93-5cc3-4adf-b3a8-655c703b2d94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:25.043972 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1156/2307] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:53:24 2022] DELETE /identity/v3/projects/25303813bbee45b1becc97f9e9a68e88 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:53:25.049048 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aed256-36ab-4a51-9e12-c058b18a6d97 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:25.049532 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aed256-36ab-4a51-9e12-c058b18a6d97 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:25.049625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aed256-36ab-4a51-9e12-c058b18a6d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:25.072645 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-b3aed256-36ab-4a51-9e12-c058b18a6d97 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:25.074174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b3aed256-36ab-4a51-9e12-c058b18a6d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:25.141097 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1152/2308] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:25.151014 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b2e85-76c6-4d4c-93e5-a46b4e483b91 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:25.151258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b2e85-76c6-4d4c-93e5-a46b4e483b91 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:25.151577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b2e85-76c6-4d4c-93e5-a46b4e483b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:25.172640 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7b2b2e85-76c6-4d4c-93e5-a46b4e483b91 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:25.173843 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7b2b2e85-76c6-4d4c-93e5-a46b4e483b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:25.182808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1157/2309] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:25.187500 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aed256-36ab-4a51-9e12-c058b18a6d97 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:25.197762 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aed256-36ab-4a51-9e12-c058b18a6d97 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:25.202902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0480f4-be49-4faa-9ce1-356b4f5fb7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:25.203515 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0480f4-be49-4faa-9ce1-356b4f5fb7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:25.203784 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0480f4-be49-4faa-9ce1-356b4f5fb7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:25.204074 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0480f4-be49-4faa-9ce1-356b4f5fb7c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:25.204729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0480f4-be49-4faa-9ce1-356b4f5fb7c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:53:25.205027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0480f4-be49-4faa-9ce1-356b4f5fb7c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:25.206657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0480f4-be49-4faa-9ce1-356b4f5fb7c1 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:25.219795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1153/2310] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:53:25.226462 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b2e85-76c6-4d4c-93e5-a46b4e483b91 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:25.240196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b2e85-76c6-4d4c-93e5-a46b4e483b91 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:25.245852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d2bbf-f1d3-4659-947d-5f6530eebb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:25.246471 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d2bbf-f1d3-4659-947d-5f6530eebb99 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:25.246749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d2bbf-f1d3-4659-947d-5f6530eebb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:25.247014 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d2bbf-f1d3-4659-947d-5f6530eebb99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:25.247751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d2bbf-f1d3-4659-947d-5f6530eebb99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:25.249731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d2bbf-f1d3-4659-947d-5f6530eebb99 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:25.284832 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1158/2311] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:53:25 2022] POST /identity/v3/projects => generated 357 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:53:25.289237 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0480f4-be49-4faa-9ce1-356b4f5fb7c1 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:25.300104 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0480f4-be49-4faa-9ce1-356b4f5fb7c1 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:25.359359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d33ac-51fc-4c2c-a057-1ec51d848962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:25.359973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d33ac-51fc-4c2c-a057-1ec51d848962 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:25.360278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d33ac-51fc-4c2c-a057-1ec51d848962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:25.360545 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d33ac-51fc-4c2c-a057-1ec51d848962 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:25.361216 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d33ac-51fc-4c2c-a057-1ec51d848962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:25.363149 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d33ac-51fc-4c2c-a057-1ec51d848962 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:25.388457 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1154/2312] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:53:25 2022] POST /identity/v3/users => generated 279 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:53:25.393786 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d2bbf-f1d3-4659-947d-5f6530eebb99 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:25.407169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d2bbf-f1d3-4659-947d-5f6530eebb99 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:25.413352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b89d98-1031-4abb-967e-80121246bfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:25.413927 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b89d98-1031-4abb-967e-80121246bfc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:25.414215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b89d98-1031-4abb-967e-80121246bfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:25.414473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b89d98-1031-4abb-967e-80121246bfc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:25.415935 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b89d98-1031-4abb-967e-80121246bfc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:25.417833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b89d98-1031-4abb-967e-80121246bfc0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:25.429828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1159/2313] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:53:25.435478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d33ac-51fc-4c2c-a057-1ec51d848962 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:25.448826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d33ac-51fc-4c2c-a057-1ec51d848962 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:25.455649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1462d0f-3552-4038-ab82-7e9645317bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:25.456468 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1462d0f-3552-4038-ab82-7e9645317bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:25.456748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1462d0f-3552-4038-ab82-7e9645317bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:25.457116 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1462d0f-3552-4038-ab82-7e9645317bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:25.458031 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1462d0f-3552-4038-ab82-7e9645317bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:25.460539 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1462d0f-3552-4038-ab82-7e9645317bd0 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:25.473670 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1155/2314] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:53:25.478308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b89d98-1031-4abb-967e-80121246bfc0 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:25.489360 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b89d98-1031-4abb-967e-80121246bfc0 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:25.495921 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f08645-9db1-4f8b-8868-f42547f6225b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:25.496616 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f08645-9db1-4f8b-8868-f42547f6225b admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:25.496889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f08645-9db1-4f8b-8868-f42547f6225b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:25.497190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f08645-9db1-4f8b-8868-f42547f6225b admin admin] PATH_INFO: `/v3/projects/f1112fcc206e4454a8a37fa58724598e/users/16c4af4bdbac4f43a56fc670dae8c39f/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:25.513633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f08645-9db1-4f8b-8868-f42547f6225b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1112fcc206e4454a8a37fa58724598e, user_id=16c4af4bdbac4f43a56fc670dae8c39f, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:25.516465 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f08645-9db1-4f8b-8868-f42547f6225b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:25.534935 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1160/2315] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:53:25 2022] PUT /identity/v3/projects/f1112fcc206e4454a8a37fa58724598e/users/16c4af4bdbac4f43a56fc670dae8c39f/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:25.550804 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c60de-dcf5-4787-aa28-f602c6cc3b63 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:25.551238 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c60de-dcf5-4787-aa28-f602c6cc3b63 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:25.551652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c60de-dcf5-4787-aa28-f602c6cc3b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:25.585762 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-fa2c60de-dcf5-4787-aa28-f602c6cc3b63 None None] MFA Rules not processed for user `16c4af4bdbac4f43a56fc670dae8c39f`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:25.586980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-fa2c60de-dcf5-4787-aa28-f602c6cc3b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:25.600860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:25.656615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1156/2316] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:25.661807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f08645-9db1-4f8b-8868-f42547f6225b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:25.701230 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:25.741621 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f08645-9db1-4f8b-8868-f42547f6225b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:25.746913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad6167-4b64-4a12-83f4-5697d59989bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:25.747519 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad6167-4b64-4a12-83f4-5697d59989bc admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:25.747825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad6167-4b64-4a12-83f4-5697d59989bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:25.748120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad6167-4b64-4a12-83f4-5697d59989bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:25.748741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad6167-4b64-4a12-83f4-5697d59989bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:25.750661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad6167-4b64-4a12-83f4-5697d59989bc admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:25.782419 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1161/2317] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:53:25 2022] POST /identity/v3/projects => generated 355 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:53:25.788838 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c60de-dcf5-4787-aa28-f602c6cc3b63 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:25.800505 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c60de-dcf5-4787-aa28-f602c6cc3b63 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:25.860639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5114b8d-d314-42b1-9a50-976c132396f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:25.861243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5114b8d-d314-42b1-9a50-976c132396f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:25.861517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5114b8d-d314-42b1-9a50-976c132396f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:25.861781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5114b8d-d314-42b1-9a50-976c132396f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:25.862423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5114b8d-d314-42b1-9a50-976c132396f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:25.864353 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5114b8d-d314-42b1-9a50-976c132396f1 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:25.887880 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1157/2318] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:53:25 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:53:25.892135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad6167-4b64-4a12-83f4-5697d59989bc admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:25.903576 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad6167-4b64-4a12-83f4-5697d59989bc admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:25.910075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d5b2e-38c2-4cd1-9fb2-37bd07ae68eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:25.910628 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d5b2e-38c2-4cd1-9fb2-37bd07ae68eb admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:25.910902 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d5b2e-38c2-4cd1-9fb2-37bd07ae68eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:25.911168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d5b2e-38c2-4cd1-9fb2-37bd07ae68eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:25.911819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d5b2e-38c2-4cd1-9fb2-37bd07ae68eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:25.913496 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d5b2e-38c2-4cd1-9fb2-37bd07ae68eb admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:25.925495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1162/2319] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:53:25.930285 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5114b8d-d314-42b1-9a50-976c132396f1 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:25.940699 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5114b8d-d314-42b1-9a50-976c132396f1 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:25.945390 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d5d10-ba74-4cdc-a332-b5ba46074001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:25.946048 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d5d10-ba74-4cdc-a332-b5ba46074001 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:25.946323 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d5d10-ba74-4cdc-a332-b5ba46074001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:25.946592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d5d10-ba74-4cdc-a332-b5ba46074001 admin admin] PATH_INFO: `/v3/projects/b517de38691643ea9b145abe84857a7e/users/7448d35711434b64843caccd6bba5d3f/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:25.961919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d5d10-ba74-4cdc-a332-b5ba46074001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b517de38691643ea9b145abe84857a7e, user_id=7448d35711434b64843caccd6bba5d3f, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:25.964863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d5d10-ba74-4cdc-a332-b5ba46074001 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:25.980730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1158/2320] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:53:25 2022] PUT /identity/v3/projects/b517de38691643ea9b145abe84857a7e/users/7448d35711434b64843caccd6bba5d3f/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:25.984837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d5b2e-38c2-4cd1-9fb2-37bd07ae68eb admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:25.994992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d5b2e-38c2-4cd1-9fb2-37bd07ae68eb admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:26.052815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32f3e7-422f-4c5d-9bed-6bfaff823402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:26.053785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32f3e7-422f-4c5d-9bed-6bfaff823402 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:26.054060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32f3e7-422f-4c5d-9bed-6bfaff823402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:26.054330 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32f3e7-422f-4c5d-9bed-6bfaff823402 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:26.054967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32f3e7-422f-4c5d-9bed-6bfaff823402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:26.056778 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32f3e7-422f-4c5d-9bed-6bfaff823402 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:26.066738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1163/2321] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:53:26.071249 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d5d10-ba74-4cdc-a332-b5ba46074001 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:26.082427 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d5d10-ba74-4cdc-a332-b5ba46074001 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:26.088158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa7c6a-a8af-400a-b131-894b5ea7364e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:26.088823 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa7c6a-a8af-400a-b131-894b5ea7364e admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:26.089093 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa7c6a-a8af-400a-b131-894b5ea7364e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:26.089360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa7c6a-a8af-400a-b131-894b5ea7364e admin admin] PATH_INFO: `/v3/projects/b517de38691643ea9b145abe84857a7e/users/7448d35711434b64843caccd6bba5d3f/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:26.091470 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aa7c6a-a8af-400a-b131-894b5ea7364e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b517de38691643ea9b145abe84857a7e, user_id=7448d35711434b64843caccd6bba5d3f, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:26.094166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aa7c6a-a8af-400a-b131-894b5ea7364e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:26.105474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1159/2322] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:53:26 2022] PUT /identity/v3/projects/b517de38691643ea9b145abe84857a7e/users/7448d35711434b64843caccd6bba5d3f/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:26.116792 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088e783-9d72-4abf-8af9-72fc2daf1bc2 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:26.117093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088e783-9d72-4abf-8af9-72fc2daf1bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:26.117359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088e783-9d72-4abf-8af9-72fc2daf1bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:26.158028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4088e783-9d72-4abf-8af9-72fc2daf1bc2 None None] MFA Rules not processed for user `7448d35711434b64843caccd6bba5d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:26.159238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4088e783-9d72-4abf-8af9-72fc2daf1bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:26.233851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1164/2323] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:26.246654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa7c6a-a8af-400a-b131-894b5ea7364e admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:26.249752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-84aa7c6a-a8af-400a-b131-894b5ea7364e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:26.326879 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa7c6a-a8af-400a-b131-894b5ea7364e admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:26.330708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080a4f7-ea4d-45cf-8f8a-d6dee046d422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:26.331318 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080a4f7-ea4d-45cf-8f8a-d6dee046d422 service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:26.331592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080a4f7-ea4d-45cf-8f8a-d6dee046d422 service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:26.331858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080a4f7-ea4d-45cf-8f8a-d6dee046d422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:26.341950 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080a4f7-ea4d-45cf-8f8a-d6dee046d422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:26.343386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080a4f7-ea4d-45cf-8f8a-d6dee046d422 service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:26.353196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1160/2324] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:53:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:53:26.433186 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088e783-9d72-4abf-8af9-72fc2daf1bc2 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:26.443608 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088e783-9d72-4abf-8af9-72fc2daf1bc2 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:26.447693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fef3f-9a91-4525-b720-ed0a1ec62d1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:26.448331 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fef3f-9a91-4525-b720-ed0a1ec62d1b service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:26.448603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fef3f-9a91-4525-b720-ed0a1ec62d1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:26.448851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fef3f-9a91-4525-b720-ed0a1ec62d1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:26.475382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:26.508084 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827fef3f-9a91-4525-b720-ed0a1ec62d1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:26.509582 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827fef3f-9a91-4525-b720-ed0a1ec62d1b service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:26.519213 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1165/2325] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:53:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:53:36.348665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=90,keystone:INSERT=5 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:36.515082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=16 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:41.865559 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080a4f7-ea4d-45cf-8f8a-d6dee046d422 service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:41.938285 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080a4f7-ea4d-45cf-8f8a-d6dee046d422 service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:41.943394 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18501914-bd4d-4e5d-8e52-142cc236ead3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:41.944056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18501914-bd4d-4e5d-8e52-142cc236ead3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:41.944333 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18501914-bd4d-4e5d-8e52-142cc236ead3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:41.944603 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18501914-bd4d-4e5d-8e52-142cc236ead3 admin admin] PATH_INFO: `/v3/users/16c4af4bdbac4f43a56fc670dae8c39f` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:41.945779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18501914-bd4d-4e5d-8e52-142cc236ead3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16c4af4bdbac4f43a56fc670dae8c39f)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:41.947700 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18501914-bd4d-4e5d-8e52-142cc236ead3 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:42.005762 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-18501914-bd4d-4e5d-8e52-142cc236ead3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16c4af4bdbac4f43a56fc670dae8c39f'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:42.005762 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-18501914-bd4d-4e5d-8e52-142cc236ead3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16c4af4bdbac4f43a56fc670dae8c39f'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:42.005762 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-18501914-bd4d-4e5d-8e52-142cc236ead3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16c4af4bdbac4f43a56fc670dae8c39f'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:42.015284 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-18501914-bd4d-4e5d-8e52-142cc236ead3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16c4af4bdbac4f43a56fc670dae8c39f'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:42.029351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-18501914-bd4d-4e5d-8e52-142cc236ead3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16c4af4bdbac4f43a56fc670dae8c39f'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:42.043183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1161/2326] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:53:41 2022] DELETE /identity/v3/users/16c4af4bdbac4f43a56fc670dae8c39f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:42.048941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c4309-2114-4a43-bf3c-8e32b5ebb350 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:42.049507 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c4309-2114-4a43-bf3c-8e32b5ebb350 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:42.049855 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c4309-2114-4a43-bf3c-8e32b5ebb350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:42.077757 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-df0c4309-2114-4a43-bf3c-8e32b5ebb350 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:42.079237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-df0c4309-2114-4a43-bf3c-8e32b5ebb350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:42.151680 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1166/2327] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:42.163655 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18501914-bd4d-4e5d-8e52-142cc236ead3 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:42.165575 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-18501914-bd4d-4e5d-8e52-142cc236ead3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:42.241120 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18501914-bd4d-4e5d-8e52-142cc236ead3 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:42.245525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4f056-88f1-440c-9ab8-baa146ad40ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:42.246146 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4f056-88f1-440c-9ab8-baa146ad40ec service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:42.246422 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4f056-88f1-440c-9ab8-baa146ad40ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:42.246689 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4f056-88f1-440c-9ab8-baa146ad40ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:42.257255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4f056-88f1-440c-9ab8-baa146ad40ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:42.258698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4f056-88f1-440c-9ab8-baa146ad40ec service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:42.269020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1162/2328] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:53:42.565249 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c4309-2114-4a43-bf3c-8e32b5ebb350 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:42.567670 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-df0c4309-2114-4a43-bf3c-8e32b5ebb350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:42.583938 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c4309-2114-4a43-bf3c-8e32b5ebb350 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:42.588503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168571fe-78d0-4dcb-b9c9-d554f3277090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:42.589147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168571fe-78d0-4dcb-b9c9-d554f3277090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:42.589420 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168571fe-78d0-4dcb-b9c9-d554f3277090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:42.589682 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168571fe-78d0-4dcb-b9c9-d554f3277090 admin admin] PATH_INFO: `/v3/projects/f1112fcc206e4454a8a37fa58724598e` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:42.591166 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168571fe-78d0-4dcb-b9c9-d554f3277090 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1112fcc206e4454a8a37fa58724598e)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:42.592740 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168571fe-78d0-4dcb-b9c9-d554f3277090 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:42.652278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-168571fe-78d0-4dcb-b9c9-d554f3277090 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1112fcc206e4454a8a37fa58724598e', 'request_id': 'req-168571fe-78d0-4dcb-b9c9-d554f3277090'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:42.662422 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-168571fe-78d0-4dcb-b9c9-d554f3277090 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1112fcc206e4454a8a37fa58724598e', 'request_id': 'req-168571fe-78d0-4dcb-b9c9-d554f3277090'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:42.670976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-168571fe-78d0-4dcb-b9c9-d554f3277090 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1112fcc206e4454a8a37fa58724598e', 'request_id': 'req-168571fe-78d0-4dcb-b9c9-d554f3277090'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:42.683751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-168571fe-78d0-4dcb-b9c9-d554f3277090 admin admin] The token cache is being invalidate because project f1112fcc206e4454a8a37fa58724598e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:53:42.684172 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-168571fe-78d0-4dcb-b9c9-d554f3277090 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:42.685142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-168571fe-78d0-4dcb-b9c9-d554f3277090 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:42.687044 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1167/2329] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:53:42 2022] DELETE /identity/v3/projects/f1112fcc206e4454a8a37fa58724598e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:42.691420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4f056-88f1-440c-9ab8-baa146ad40ec service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:42.703147 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4f056-88f1-440c-9ab8-baa146ad40ec service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:42.723528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:42.778166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba65b27-86ea-4c5b-af7b-398f0f3d24e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:42.778811 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba65b27-86ea-4c5b-af7b-398f0f3d24e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:42.779089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba65b27-86ea-4c5b-af7b-398f0f3d24e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:42.779365 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba65b27-86ea-4c5b-af7b-398f0f3d24e1 admin admin] PATH_INFO: `/v3/users/7448d35711434b64843caccd6bba5d3f` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:42.780553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba65b27-86ea-4c5b-af7b-398f0f3d24e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7448d35711434b64843caccd6bba5d3f)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:42.782564 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba65b27-86ea-4c5b-af7b-398f0f3d24e1 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:42.832311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eba65b27-86ea-4c5b-af7b-398f0f3d24e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7448d35711434b64843caccd6bba5d3f'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:42.832962 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eba65b27-86ea-4c5b-af7b-398f0f3d24e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7448d35711434b64843caccd6bba5d3f'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:42.833686 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eba65b27-86ea-4c5b-af7b-398f0f3d24e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7448d35711434b64843caccd6bba5d3f'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:42.843073 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eba65b27-86ea-4c5b-af7b-398f0f3d24e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7448d35711434b64843caccd6bba5d3f'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:42.856725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eba65b27-86ea-4c5b-af7b-398f0f3d24e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7448d35711434b64843caccd6bba5d3f'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:42.869940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1163/2330] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:53:42 2022] DELETE /identity/v3/users/7448d35711434b64843caccd6bba5d3f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:43.176645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168571fe-78d0-4dcb-b9c9-d554f3277090 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:43.179319 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-168571fe-78d0-4dcb-b9c9-d554f3277090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:43.200389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168571fe-78d0-4dcb-b9c9-d554f3277090 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:43.269241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc104b-17e1-4b35-9f88-7f69d2bfa737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:43.269843 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc104b-17e1-4b35-9f88-7f69d2bfa737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:43.270116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc104b-17e1-4b35-9f88-7f69d2bfa737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:43.270541 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc104b-17e1-4b35-9f88-7f69d2bfa737 admin admin] PATH_INFO: `/v3/projects/b517de38691643ea9b145abe84857a7e` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:43.271803 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc104b-17e1-4b35-9f88-7f69d2bfa737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b517de38691643ea9b145abe84857a7e)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:43.273821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc104b-17e1-4b35-9f88-7f69d2bfa737 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:43.310452 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:43.331172 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-75cc104b-17e1-4b35-9f88-7f69d2bfa737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b517de38691643ea9b145abe84857a7e', 'request_id': 'req-75cc104b-17e1-4b35-9f88-7f69d2bfa737'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:43.340361 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-75cc104b-17e1-4b35-9f88-7f69d2bfa737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b517de38691643ea9b145abe84857a7e', 'request_id': 'req-75cc104b-17e1-4b35-9f88-7f69d2bfa737'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:43.347042 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-75cc104b-17e1-4b35-9f88-7f69d2bfa737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b517de38691643ea9b145abe84857a7e', 'request_id': 'req-75cc104b-17e1-4b35-9f88-7f69d2bfa737'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:43.360633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-75cc104b-17e1-4b35-9f88-7f69d2bfa737 admin admin] The token cache is being invalidate because project b517de38691643ea9b145abe84857a7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:53:43.360926 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-75cc104b-17e1-4b35-9f88-7f69d2bfa737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:43.361668 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-75cc104b-17e1-4b35-9f88-7f69d2bfa737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:43.363282 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1168/2331] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:53:43 2022] DELETE /identity/v3/projects/b517de38691643ea9b145abe84857a7e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:43.374693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2691b8-6eda-42ec-a6ef-3cf554122939 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:43.375042 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2691b8-6eda-42ec-a6ef-3cf554122939 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:43.375482 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2691b8-6eda-42ec-a6ef-3cf554122939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:43.398542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-ff2691b8-6eda-42ec-a6ef-3cf554122939 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:43.399768 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-ff2691b8-6eda-42ec-a6ef-3cf554122939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:43.466722 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1164/2332] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:43.474861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3132f-04b9-4b95-b348-40176d8a32ed None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:43.475120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3132f-04b9-4b95-b348-40176d8a32ed None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:43.475438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3132f-04b9-4b95-b348-40176d8a32ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:43.500095 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-6ba3132f-04b9-4b95-b348-40176d8a32ed None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:43.501348 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6ba3132f-04b9-4b95-b348-40176d8a32ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:43.510976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1169/2333] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:43.516151 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2691b8-6eda-42ec-a6ef-3cf554122939 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:43.526882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2691b8-6eda-42ec-a6ef-3cf554122939 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:43.531554 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305685f-80b0-473b-9488-6780a988d78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:43.532211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305685f-80b0-473b-9488-6780a988d78b admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:43.532483 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305685f-80b0-473b-9488-6780a988d78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:43.532746 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305685f-80b0-473b-9488-6780a988d78b admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:43.533395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305685f-80b0-473b-9488-6780a988d78b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:53:43.533689 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305685f-80b0-473b-9488-6780a988d78b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:43.535563 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305685f-80b0-473b-9488-6780a988d78b admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:43.553101 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1165/2334] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:53:43.554924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3132f-04b9-4b95-b348-40176d8a32ed None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:43.565151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3132f-04b9-4b95-b348-40176d8a32ed None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:43.571859 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516277c-167e-498e-8b51-ab957b6be068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:43.573461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516277c-167e-498e-8b51-ab957b6be068 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:43.573461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516277c-167e-498e-8b51-ab957b6be068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:43.573461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516277c-167e-498e-8b51-ab957b6be068 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:43.573804 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516277c-167e-498e-8b51-ab957b6be068 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:43.575692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516277c-167e-498e-8b51-ab957b6be068 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:43.608467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1170/2335] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:53:43 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:53:43.612889 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305685f-80b0-473b-9488-6780a988d78b admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:43.626001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305685f-80b0-473b-9488-6780a988d78b admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:43.665587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:43.703853 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb280d-4ad4-4614-b16a-55f50094956a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:43.704659 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb280d-4ad4-4614-b16a-55f50094956a admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:43.705032 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb280d-4ad4-4614-b16a-55f50094956a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:43.705397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb280d-4ad4-4614-b16a-55f50094956a admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:43.706138 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb280d-4ad4-4614-b16a-55f50094956a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:43.709208 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb280d-4ad4-4614-b16a-55f50094956a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:43.733464 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1166/2336] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:53:43 2022] POST /identity/v3/users => generated 282 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:53:43.739301 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516277c-167e-498e-8b51-ab957b6be068 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:43.751309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516277c-167e-498e-8b51-ab957b6be068 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:43.756732 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc3055-46c0-4e31-99c9-8c6379df5a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:43.757586 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc3055-46c0-4e31-99c9-8c6379df5a27 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:43.757829 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc3055-46c0-4e31-99c9-8c6379df5a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:43.758099 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc3055-46c0-4e31-99c9-8c6379df5a27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:43.758747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc3055-46c0-4e31-99c9-8c6379df5a27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:43.760447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc3055-46c0-4e31-99c9-8c6379df5a27 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:43.776107 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1171/2337] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:53:43.778189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb280d-4ad4-4614-b16a-55f50094956a admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:43.794174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb280d-4ad4-4614-b16a-55f50094956a admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:43.805951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13596f40-9c56-4ce8-90ab-f32f000defa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:43.807579 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13596f40-9c56-4ce8-90ab-f32f000defa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:43.807926 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13596f40-9c56-4ce8-90ab-f32f000defa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:43.808411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13596f40-9c56-4ce8-90ab-f32f000defa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:43.809195 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13596f40-9c56-4ce8-90ab-f32f000defa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:43.811387 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13596f40-9c56-4ce8-90ab-f32f000defa7 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:43.825891 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1167/2338] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:53:43.830544 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc3055-46c0-4e31-99c9-8c6379df5a27 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:43.843749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc3055-46c0-4e31-99c9-8c6379df5a27 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:43.849427 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec03b13-7861-4a9c-afc3-7a63a00fd779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:43.850102 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec03b13-7861-4a9c-afc3-7a63a00fd779 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:43.850392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec03b13-7861-4a9c-afc3-7a63a00fd779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:43.850654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec03b13-7861-4a9c-afc3-7a63a00fd779 admin admin] PATH_INFO: `/v3/projects/97aa4708aecb4f0ea8c917293edf7caf/users/c33f97a8f0a9461a9a60fce86009b210/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:43.866942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec03b13-7861-4a9c-afc3-7a63a00fd779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97aa4708aecb4f0ea8c917293edf7caf, user_id=c33f97a8f0a9461a9a60fce86009b210, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:43.869668 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec03b13-7861-4a9c-afc3-7a63a00fd779 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:43.884156 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1172/2339] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:53:43 2022] PUT /identity/v3/projects/97aa4708aecb4f0ea8c917293edf7caf/users/c33f97a8f0a9461a9a60fce86009b210/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:53:43.896150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac7ff1-ff70-42c5-a454-7eb5d253d6a3 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:43.896645 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac7ff1-ff70-42c5-a454-7eb5d253d6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:43.896783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac7ff1-ff70-42c5-a454-7eb5d253d6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:43.931053 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-4eac7ff1-ff70-42c5-a454-7eb5d253d6a3 None None] MFA Rules not processed for user `c33f97a8f0a9461a9a60fce86009b210`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:43.933000 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4eac7ff1-ff70-42c5-a454-7eb5d253d6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:44.011173 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1168/2340] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:53:44.015859 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec03b13-7861-4a9c-afc3-7a63a00fd779 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:44.084806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec03b13-7861-4a9c-afc3-7a63a00fd779 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:44.089784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dce5af-c0c8-4c0b-ac59-dd8fec37a403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:44.090426 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dce5af-c0c8-4c0b-ac59-dd8fec37a403 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:44.090705 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dce5af-c0c8-4c0b-ac59-dd8fec37a403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:44.090973 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dce5af-c0c8-4c0b-ac59-dd8fec37a403 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:44.091613 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dce5af-c0c8-4c0b-ac59-dd8fec37a403 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:44.093584 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dce5af-c0c8-4c0b-ac59-dd8fec37a403 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:44.130196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1173/2341] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:53:44 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:53:44.134536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac7ff1-ff70-42c5-a454-7eb5d253d6a3 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:44.148362 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac7ff1-ff70-42c5-a454-7eb5d253d6a3 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:44.212696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcbad6-eb1b-4704-b47f-b6d76a68c59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:44.213554 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcbad6-eb1b-4704-b47f-b6d76a68c59e admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:44.213927 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcbad6-eb1b-4704-b47f-b6d76a68c59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:44.214346 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcbad6-eb1b-4704-b47f-b6d76a68c59e admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:44.215031 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcbad6-eb1b-4704-b47f-b6d76a68c59e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:44.218016 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcbad6-eb1b-4704-b47f-b6d76a68c59e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:44.248491 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1169/2342] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:53:44 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:53:44.253124 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dce5af-c0c8-4c0b-ac59-dd8fec37a403 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:44.267241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dce5af-c0c8-4c0b-ac59-dd8fec37a403 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:44.273651 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3821f34-b42f-46d8-b732-a5e199bdf226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:44.274410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3821f34-b42f-46d8-b732-a5e199bdf226 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:44.274677 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3821f34-b42f-46d8-b732-a5e199bdf226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:44.274943 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3821f34-b42f-46d8-b732-a5e199bdf226 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:44.275589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3821f34-b42f-46d8-b732-a5e199bdf226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:44.277232 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3821f34-b42f-46d8-b732-a5e199bdf226 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:44.288643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1174/2343] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:53:44.293019 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcbad6-eb1b-4704-b47f-b6d76a68c59e admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:44.303574 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcbad6-eb1b-4704-b47f-b6d76a68c59e admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:44.308388 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eeedb5-3e77-435a-a7b5-4ebda59f99d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:44.309045 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eeedb5-3e77-435a-a7b5-4ebda59f99d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:44.309416 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eeedb5-3e77-435a-a7b5-4ebda59f99d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:44.309586 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eeedb5-3e77-435a-a7b5-4ebda59f99d8 admin admin] PATH_INFO: `/v3/projects/e5867ef27c8e412cae6805afb0ae7f60/users/00ff9c486dad46ee9e759d937590c129/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:44.325148 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eeedb5-3e77-435a-a7b5-4ebda59f99d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5867ef27c8e412cae6805afb0ae7f60, user_id=00ff9c486dad46ee9e759d937590c129, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:44.327794 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eeedb5-3e77-435a-a7b5-4ebda59f99d8 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:44.346927 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1170/2344] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:53:44 2022] PUT /identity/v3/projects/e5867ef27c8e412cae6805afb0ae7f60/users/00ff9c486dad46ee9e759d937590c129/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:44.352258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3821f34-b42f-46d8-b732-a5e199bdf226 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:44.366973 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3821f34-b42f-46d8-b732-a5e199bdf226 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:44.389486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:44.440026 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d1dec-daaa-40be-972c-82f909d40524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:44.440635 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d1dec-daaa-40be-972c-82f909d40524 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:44.440915 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d1dec-daaa-40be-972c-82f909d40524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:44.441216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d1dec-daaa-40be-972c-82f909d40524 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:44.442118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d1dec-daaa-40be-972c-82f909d40524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:44.443750 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d1dec-daaa-40be-972c-82f909d40524 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:44.453905 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1175/2345] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:53:44.458885 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eeedb5-3e77-435a-a7b5-4ebda59f99d8 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:44.467770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:44.474477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eeedb5-3e77-435a-a7b5-4ebda59f99d8 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:44.479666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fa4c5-c5ba-42de-bf7f-5dc7c7cc216b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:44.480355 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fa4c5-c5ba-42de-bf7f-5dc7c7cc216b admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:44.480651 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fa4c5-c5ba-42de-bf7f-5dc7c7cc216b admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:44.480918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fa4c5-c5ba-42de-bf7f-5dc7c7cc216b admin admin] PATH_INFO: `/v3/projects/e5867ef27c8e412cae6805afb0ae7f60/users/00ff9c486dad46ee9e759d937590c129/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:44.482814 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fa4c5-c5ba-42de-bf7f-5dc7c7cc216b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5867ef27c8e412cae6805afb0ae7f60, user_id=00ff9c486dad46ee9e759d937590c129, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:44.485539 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fa4c5-c5ba-42de-bf7f-5dc7c7cc216b admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:44.496375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1171/2346] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:53:44 2022] PUT /identity/v3/projects/e5867ef27c8e412cae6805afb0ae7f60/users/00ff9c486dad46ee9e759d937590c129/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:44.506989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188e857-89bf-44d4-9f0d-3f21ca59ae42 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:44.507423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188e857-89bf-44d4-9f0d-3f21ca59ae42 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:44.507741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188e857-89bf-44d4-9f0d-3f21ca59ae42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:44.540882 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8188e857-89bf-44d4-9f0d-3f21ca59ae42 None None] MFA Rules not processed for user `00ff9c486dad46ee9e759d937590c129`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:44.542123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8188e857-89bf-44d4-9f0d-3f21ca59ae42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:44.618824 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1176/2347] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:44.630936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fa4c5-c5ba-42de-bf7f-5dc7c7cc216b admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:44.633024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-9a5fa4c5-c5ba-42de-bf7f-5dc7c7cc216b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:44.693145 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fa4c5-c5ba-42de-bf7f-5dc7c7cc216b admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:44.697624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db47d6d-1043-48b7-bd55-6098451170fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:44.698400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db47d6d-1043-48b7-bd55-6098451170fa service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:44.698678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db47d6d-1043-48b7-bd55-6098451170fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:44.698967 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db47d6d-1043-48b7-bd55-6098451170fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:44.711895 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db47d6d-1043-48b7-bd55-6098451170fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:44.714070 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db47d6d-1043-48b7-bd55-6098451170fa service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:44.727734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1172/2348] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:53:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:53:44.812021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188e857-89bf-44d4-9f0d-3f21ca59ae42 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:44.824188 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188e857-89bf-44d4-9f0d-3f21ca59ae42 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:44.828584 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c9351-69bb-44c9-8689-10e97ecdd233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:44.829209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c9351-69bb-44c9-8689-10e97ecdd233 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:44.829472 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c9351-69bb-44c9-8689-10e97ecdd233 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:44.829751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c9351-69bb-44c9-8689-10e97ecdd233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:44.893932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c9351-69bb-44c9-8689-10e97ecdd233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:44.895534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c9351-69bb-44c9-8689-10e97ecdd233 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:44.905657 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1177/2349] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:53:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:53:54.722580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:54.901572 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=80 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:56.565571 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db47d6d-1043-48b7-bd55-6098451170fa service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:56.636471 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db47d6d-1043-48b7-bd55-6098451170fa service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:56.641156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985d358-6c3f-4290-918a-052ec41b0ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:56.641785 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985d358-6c3f-4290-918a-052ec41b0ffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:56.642130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985d358-6c3f-4290-918a-052ec41b0ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:56.642336 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985d358-6c3f-4290-918a-052ec41b0ffb admin admin] PATH_INFO: `/v3/users/c33f97a8f0a9461a9a60fce86009b210` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:56.643450 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5985d358-6c3f-4290-918a-052ec41b0ffb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c33f97a8f0a9461a9a60fce86009b210)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:56.645454 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5985d358-6c3f-4290-918a-052ec41b0ffb admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:56.695215 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5985d358-6c3f-4290-918a-052ec41b0ffb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c33f97a8f0a9461a9a60fce86009b210'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:56.695858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5985d358-6c3f-4290-918a-052ec41b0ffb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c33f97a8f0a9461a9a60fce86009b210'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:56.696495 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5985d358-6c3f-4290-918a-052ec41b0ffb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c33f97a8f0a9461a9a60fce86009b210'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:56.705679 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5985d358-6c3f-4290-918a-052ec41b0ffb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c33f97a8f0a9461a9a60fce86009b210'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:56.721181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5985d358-6c3f-4290-918a-052ec41b0ffb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c33f97a8f0a9461a9a60fce86009b210'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:56.734511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1173/2350] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:53:56 2022] DELETE /identity/v3/users/c33f97a8f0a9461a9a60fce86009b210 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:56.739486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf0757-47c8-479f-b989-6ac67b13d85d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:56.739984 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf0757-47c8-479f-b989-6ac67b13d85d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:56.740110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf0757-47c8-479f-b989-6ac67b13d85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:56.760907 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8ebf0757-47c8-479f-b989-6ac67b13d85d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:56.762141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8ebf0757-47c8-479f-b989-6ac67b13d85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:56.831459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1178/2351] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:56.842171 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985d358-6c3f-4290-918a-052ec41b0ffb admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:56.844403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5985d358-6c3f-4290-918a-052ec41b0ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:56.918650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985d358-6c3f-4290-918a-052ec41b0ffb admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:56.922818 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b283f-94b6-404e-a277-e2baa5a4f74e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:56.923507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b283f-94b6-404e-a277-e2baa5a4f74e service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:56.923777 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b283f-94b6-404e-a277-e2baa5a4f74e service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:56.924039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b283f-94b6-404e-a277-e2baa5a4f74e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:56.935040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b283f-94b6-404e-a277-e2baa5a4f74e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:56.936491 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b283f-94b6-404e-a277-e2baa5a4f74e service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:56.946889 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1174/2352] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:53:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:53:57.291061 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf0757-47c8-479f-b989-6ac67b13d85d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:57.293398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8ebf0757-47c8-479f-b989-6ac67b13d85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:57.309032 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf0757-47c8-479f-b989-6ac67b13d85d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:57.314546 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84732d-0bd9-4560-a954-007b22af01fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:57.315306 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84732d-0bd9-4560-a954-007b22af01fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:57.315577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84732d-0bd9-4560-a954-007b22af01fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:57.315986 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84732d-0bd9-4560-a954-007b22af01fd admin admin] PATH_INFO: `/v3/projects/97aa4708aecb4f0ea8c917293edf7caf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:57.317115 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84732d-0bd9-4560-a954-007b22af01fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=97aa4708aecb4f0ea8c917293edf7caf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:57.319152 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84732d-0bd9-4560-a954-007b22af01fd admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:57.373277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8a84732d-0bd9-4560-a954-007b22af01fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97aa4708aecb4f0ea8c917293edf7caf', 'request_id': 'req-8a84732d-0bd9-4560-a954-007b22af01fd'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:57.382577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8a84732d-0bd9-4560-a954-007b22af01fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97aa4708aecb4f0ea8c917293edf7caf', 'request_id': 'req-8a84732d-0bd9-4560-a954-007b22af01fd'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:57.389065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8a84732d-0bd9-4560-a954-007b22af01fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97aa4708aecb4f0ea8c917293edf7caf', 'request_id': 'req-8a84732d-0bd9-4560-a954-007b22af01fd'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:57.400257 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8a84732d-0bd9-4560-a954-007b22af01fd admin admin] The token cache is being invalidate because project 97aa4708aecb4f0ea8c917293edf7caf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:53:57.400544 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8a84732d-0bd9-4560-a954-007b22af01fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:57.401175 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8a84732d-0bd9-4560-a954-007b22af01fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:57.402675 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1179/2353] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:53:57 2022] DELETE /identity/v3/projects/97aa4708aecb4f0ea8c917293edf7caf => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:57.407080 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b283f-94b6-404e-a277-e2baa5a4f74e service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:57.417648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b283f-94b6-404e-a277-e2baa5a4f74e service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:57.439592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:57.484110 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18280d-eb00-43d6-9617-312933a974fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:57.484601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18280d-eb00-43d6-9617-312933a974fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:57.484879 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18280d-eb00-43d6-9617-312933a974fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:57.485145 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18280d-eb00-43d6-9617-312933a974fa admin admin] PATH_INFO: `/v3/users/00ff9c486dad46ee9e759d937590c129` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:57.486412 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18280d-eb00-43d6-9617-312933a974fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=00ff9c486dad46ee9e759d937590c129)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:57.488708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18280d-eb00-43d6-9617-312933a974fa admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:57.553037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-9b18280d-eb00-43d6-9617-312933a974fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00ff9c486dad46ee9e759d937590c129'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:57.554837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-9b18280d-eb00-43d6-9617-312933a974fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00ff9c486dad46ee9e759d937590c129'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:57.555881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-9b18280d-eb00-43d6-9617-312933a974fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00ff9c486dad46ee9e759d937590c129'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:57.568406 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-9b18280d-eb00-43d6-9617-312933a974fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00ff9c486dad46ee9e759d937590c129'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:57.585540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-9b18280d-eb00-43d6-9617-312933a974fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00ff9c486dad46ee9e759d937590c129'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:57.597982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1175/2354] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:53:57 2022] DELETE /identity/v3/users/00ff9c486dad46ee9e759d937590c129 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:57.894898 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84732d-0bd9-4560-a954-007b22af01fd admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:57.897967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8a84732d-0bd9-4560-a954-007b22af01fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:57.915158 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84732d-0bd9-4560-a954-007b22af01fd admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:57.983040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101b8ea-79bb-4993-8975-b76f1b98f106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:57.983691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101b8ea-79bb-4993-8975-b76f1b98f106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:57.983984 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101b8ea-79bb-4993-8975-b76f1b98f106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:57.984291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101b8ea-79bb-4993-8975-b76f1b98f106 admin admin] PATH_INFO: `/v3/projects/e5867ef27c8e412cae6805afb0ae7f60` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:57.985601 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101b8ea-79bb-4993-8975-b76f1b98f106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5867ef27c8e412cae6805afb0ae7f60)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:57.987686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101b8ea-79bb-4993-8975-b76f1b98f106 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:58.026461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:58.045116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f101b8ea-79bb-4993-8975-b76f1b98f106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5867ef27c8e412cae6805afb0ae7f60', 'request_id': 'req-f101b8ea-79bb-4993-8975-b76f1b98f106'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:58.053938 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f101b8ea-79bb-4993-8975-b76f1b98f106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5867ef27c8e412cae6805afb0ae7f60', 'request_id': 'req-f101b8ea-79bb-4993-8975-b76f1b98f106'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:58.059656 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f101b8ea-79bb-4993-8975-b76f1b98f106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5867ef27c8e412cae6805afb0ae7f60', 'request_id': 'req-f101b8ea-79bb-4993-8975-b76f1b98f106'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:58.071312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f101b8ea-79bb-4993-8975-b76f1b98f106 admin admin] The token cache is being invalidate because project e5867ef27c8e412cae6805afb0ae7f60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:53:58.071597 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f101b8ea-79bb-4993-8975-b76f1b98f106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:58.072362 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f101b8ea-79bb-4993-8975-b76f1b98f106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:58.073962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1180/2355] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:53:57 2022] DELETE /identity/v3/projects/e5867ef27c8e412cae6805afb0ae7f60 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:58.079746 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9735c12-2894-4c83-87eb-06378b2e2935 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:58.080223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9735c12-2894-4c83-87eb-06378b2e2935 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:58.080521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9735c12-2894-4c83-87eb-06378b2e2935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:58.106826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-e9735c12-2894-4c83-87eb-06378b2e2935 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:58.108052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e9735c12-2894-4c83-87eb-06378b2e2935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:58.173469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1176/2356] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:58.182840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d9b48-0731-4eda-8370-f87d360db99a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:58.183173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d9b48-0731-4eda-8370-f87d360db99a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:58.183571 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d9b48-0731-4eda-8370-f87d360db99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:58.208299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-129d9b48-0731-4eda-8370-f87d360db99a None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:58.209522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-129d9b48-0731-4eda-8370-f87d360db99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:58.220424 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1181/2357] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:58.224926 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9735c12-2894-4c83-87eb-06378b2e2935 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:58.235601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9735c12-2894-4c83-87eb-06378b2e2935 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:58.241559 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded705c-5967-4561-8b12-9a28e05aba1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:58.242265 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded705c-5967-4561-8b12-9a28e05aba1c admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:58.242547 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded705c-5967-4561-8b12-9a28e05aba1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:58.242817 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded705c-5967-4561-8b12-9a28e05aba1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:58.243506 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded705c-5967-4561-8b12-9a28e05aba1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:53:58.243804 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded705c-5967-4561-8b12-9a28e05aba1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:58.245442 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded705c-5967-4561-8b12-9a28e05aba1c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:58.258651 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1177/2358] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:53:58.263214 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d9b48-0731-4eda-8370-f87d360db99a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:58.273084 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d9b48-0731-4eda-8370-f87d360db99a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:58.278998 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df89f5-c8c7-4f2a-8499-9f726b365819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:58.279690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df89f5-c8c7-4f2a-8499-9f726b365819 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:58.279837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df89f5-c8c7-4f2a-8499-9f726b365819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:58.280119 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df89f5-c8c7-4f2a-8499-9f726b365819 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:58.280756 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df89f5-c8c7-4f2a-8499-9f726b365819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:58.282867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df89f5-c8c7-4f2a-8499-9f726b365819 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:58.324917 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1182/2359] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:53:58 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:53:58.329181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded705c-5967-4561-8b12-9a28e05aba1c admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:58.340094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded705c-5967-4561-8b12-9a28e05aba1c admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:58.367097 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:58.401769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a68654-efb7-4a6c-a2d4-2f3a3af7466c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:58.402382 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a68654-efb7-4a6c-a2d4-2f3a3af7466c admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:58.402650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a68654-efb7-4a6c-a2d4-2f3a3af7466c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:58.402920 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a68654-efb7-4a6c-a2d4-2f3a3af7466c admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:58.403553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a68654-efb7-4a6c-a2d4-2f3a3af7466c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:58.405479 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a68654-efb7-4a6c-a2d4-2f3a3af7466c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:58.429033 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1178/2360] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:53:58 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:53:58.433437 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df89f5-c8c7-4f2a-8499-9f726b365819 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:58.444180 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df89f5-c8c7-4f2a-8499-9f726b365819 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:58.448932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664f758-e8b4-47be-81dc-84cd5a5b346f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:58.449548 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664f758-e8b4-47be-81dc-84cd5a5b346f admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:58.449823 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664f758-e8b4-47be-81dc-84cd5a5b346f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:58.450087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664f758-e8b4-47be-81dc-84cd5a5b346f admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:58.450766 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664f758-e8b4-47be-81dc-84cd5a5b346f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:58.452410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664f758-e8b4-47be-81dc-84cd5a5b346f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:58.462886 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1183/2361] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:53:58.469696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a68654-efb7-4a6c-a2d4-2f3a3af7466c admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:58.486376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a68654-efb7-4a6c-a2d4-2f3a3af7466c admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:58.493983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d27947-4ed5-41cd-bb71-e4f756fa9cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:58.494595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d27947-4ed5-41cd-bb71-e4f756fa9cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:58.494875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d27947-4ed5-41cd-bb71-e4f756fa9cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:58.495128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d27947-4ed5-41cd-bb71-e4f756fa9cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:58.495791 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d27947-4ed5-41cd-bb71-e4f756fa9cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:58.497451 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d27947-4ed5-41cd-bb71-e4f756fa9cf1 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:58.509107 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1179/2362] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:53:58.514965 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664f758-e8b4-47be-81dc-84cd5a5b346f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:58.526184 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664f758-e8b4-47be-81dc-84cd5a5b346f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:58.531385 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54bfca-022d-4e9a-9fc7-41720b5976b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:58.531980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54bfca-022d-4e9a-9fc7-41720b5976b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:58.532289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54bfca-022d-4e9a-9fc7-41720b5976b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:58.532565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54bfca-022d-4e9a-9fc7-41720b5976b5 admin admin] PATH_INFO: `/v3/projects/8c72c35790994772ab4670f4a4ce0629/users/738e0cc9cec9461fbf5c602b1c994910/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:58.549841 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54bfca-022d-4e9a-9fc7-41720b5976b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c72c35790994772ab4670f4a4ce0629, user_id=738e0cc9cec9461fbf5c602b1c994910, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:58.554183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54bfca-022d-4e9a-9fc7-41720b5976b5 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:58.572580 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1184/2363] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:53:58 2022] PUT /identity/v3/projects/8c72c35790994772ab4670f4a4ce0629/users/738e0cc9cec9461fbf5c602b1c994910/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:58.588710 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51a5f2-53d9-4a53-884a-308b49a79f6f None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:58.589255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51a5f2-53d9-4a53-884a-308b49a79f6f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:58.589255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51a5f2-53d9-4a53-884a-308b49a79f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:58.620601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-dc51a5f2-53d9-4a53-884a-308b49a79f6f None None] MFA Rules not processed for user `738e0cc9cec9461fbf5c602b1c994910`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:58.621826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-dc51a5f2-53d9-4a53-884a-308b49a79f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:58.691667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1180/2364] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:58.704266 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54bfca-022d-4e9a-9fc7-41720b5976b5 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:58.706145 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ec54bfca-022d-4e9a-9fc7-41720b5976b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:58.769033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54bfca-022d-4e9a-9fc7-41720b5976b5 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:58.773707 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26764f5e-d510-4c29-b33f-3f12c7359299 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:58.774328 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26764f5e-d510-4c29-b33f-3f12c7359299 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:58.774608 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26764f5e-d510-4c29-b33f-3f12c7359299 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:58.774876 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26764f5e-d510-4c29-b33f-3f12c7359299 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:58.784050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26764f5e-d510-4c29-b33f-3f12c7359299 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:58.785443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26764f5e-d510-4c29-b33f-3f12c7359299 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:58.797785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1185/2365] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:53:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:53:58.824845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51a5f2-53d9-4a53-884a-308b49a79f6f None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:58.899273 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51a5f2-53d9-4a53-884a-308b49a79f6f None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:58.904481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e465d5-0b0e-4463-89c0-3e11c2c5b414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:58.905127 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e465d5-0b0e-4463-89c0-3e11c2c5b414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:58.905400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e465d5-0b0e-4463-89c0-3e11c2c5b414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:58.905654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e465d5-0b0e-4463-89c0-3e11c2c5b414 admin admin] PATH_INFO: `/v3/users/738e0cc9cec9461fbf5c602b1c994910` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:58.912089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e465d5-0b0e-4463-89c0-3e11c2c5b414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=738e0cc9cec9461fbf5c602b1c994910)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:58.912089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e465d5-0b0e-4463-89c0-3e11c2c5b414 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:58.954626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:58.961701 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-83e465d5-0b0e-4463-89c0-3e11c2c5b414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '738e0cc9cec9461fbf5c602b1c994910'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:58.963083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-83e465d5-0b0e-4463-89c0-3e11c2c5b414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '738e0cc9cec9461fbf5c602b1c994910'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:58.963649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-83e465d5-0b0e-4463-89c0-3e11c2c5b414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '738e0cc9cec9461fbf5c602b1c994910'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:58.975744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-83e465d5-0b0e-4463-89c0-3e11c2c5b414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '738e0cc9cec9461fbf5c602b1c994910'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:58.991649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-83e465d5-0b0e-4463-89c0-3e11c2c5b414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '738e0cc9cec9461fbf5c602b1c994910'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:59.009687 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1181/2366] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:53:58 2022] DELETE /identity/v3/users/738e0cc9cec9461fbf5c602b1c994910 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:59.014537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367cfb6-9074-49eb-97c8-3aae5661ed1c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:59.015065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367cfb6-9074-49eb-97c8-3aae5661ed1c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:59.015065 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367cfb6-9074-49eb-97c8-3aae5661ed1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:59.036870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-b367cfb6-9074-49eb-97c8-3aae5661ed1c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:59.038149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b367cfb6-9074-49eb-97c8-3aae5661ed1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:59.081560 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:59.104633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1186/2367] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:59.114555 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e465d5-0b0e-4463-89c0-3e11c2c5b414 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:59.116027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-83e465d5-0b0e-4463-89c0-3e11c2c5b414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:59.175621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e465d5-0b0e-4463-89c0-3e11c2c5b414 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:59.180125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f291ff1-642b-47c2-9e69-190305ab6bed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:59.180830 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f291ff1-642b-47c2-9e69-190305ab6bed service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:59.181107 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f291ff1-642b-47c2-9e69-190305ab6bed service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:59.181378 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f291ff1-642b-47c2-9e69-190305ab6bed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:59.191415 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f291ff1-642b-47c2-9e69-190305ab6bed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:59.192900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f291ff1-642b-47c2-9e69-190305ab6bed service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:59.203516 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1182/2368] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:53:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:53:59.540870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367cfb6-9074-49eb-97c8-3aae5661ed1c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:59.542867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b367cfb6-9074-49eb-97c8-3aae5661ed1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:59.556543 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367cfb6-9074-49eb-97c8-3aae5661ed1c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:59.561890 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ae62a-6fe7-4d15-bbb9-84c7d9255384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:59.562568 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ae62a-6fe7-4d15-bbb9-84c7d9255384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:59.562845 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ae62a-6fe7-4d15-bbb9-84c7d9255384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:59.563113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ae62a-6fe7-4d15-bbb9-84c7d9255384 admin admin] PATH_INFO: `/v3/projects/8c72c35790994772ab4670f4a4ce0629` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:59.564344 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ae62a-6fe7-4d15-bbb9-84c7d9255384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c72c35790994772ab4670f4a4ce0629)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:59.566715 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ae62a-6fe7-4d15-bbb9-84c7d9255384 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:59.631603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-152ae62a-6fe7-4d15-bbb9-84c7d9255384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c72c35790994772ab4670f4a4ce0629', 'request_id': 'req-152ae62a-6fe7-4d15-bbb9-84c7d9255384'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:59.640279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-152ae62a-6fe7-4d15-bbb9-84c7d9255384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c72c35790994772ab4670f4a4ce0629', 'request_id': 'req-152ae62a-6fe7-4d15-bbb9-84c7d9255384'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:59.646279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-152ae62a-6fe7-4d15-bbb9-84c7d9255384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c72c35790994772ab4670f4a4ce0629', 'request_id': 'req-152ae62a-6fe7-4d15-bbb9-84c7d9255384'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:59.658262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-152ae62a-6fe7-4d15-bbb9-84c7d9255384 admin admin] The token cache is being invalidate because project 8c72c35790994772ab4670f4a4ce0629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:53:59.658545 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-152ae62a-6fe7-4d15-bbb9-84c7d9255384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:59.659192 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-152ae62a-6fe7-4d15-bbb9-84c7d9255384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:59.660730 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1187/2369] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:53:59 2022] DELETE /identity/v3/projects/8c72c35790994772ab4670f4a4ce0629 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:59.666183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db9222-cc30-4cbd-abd0-49c8daacab06 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:59.666730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db9222-cc30-4cbd-abd0-49c8daacab06 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:59.666730 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db9222-cc30-4cbd-abd0-49c8daacab06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:59.689123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-70db9222-cc30-4cbd-abd0-49c8daacab06 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:59.690374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-70db9222-cc30-4cbd-abd0-49c8daacab06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:59.760318 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1183/2370] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:59.768854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a2f44-bc58-4be0-aa5f-784d253d7e22 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:59.769258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a2f44-bc58-4be0-aa5f-784d253d7e22 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:59.769530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a2f44-bc58-4be0-aa5f-784d253d7e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:59.792892 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ea3a2f44-bc58-4be0-aa5f-784d253d7e22 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:59.794108 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ea3a2f44-bc58-4be0-aa5f-784d253d7e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:59.802865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1188/2371] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:59.807844 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db9222-cc30-4cbd-abd0-49c8daacab06 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:59.817879 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db9222-cc30-4cbd-abd0-49c8daacab06 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:59.823258 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09319d-ac00-49c1-b5b9-eb880c49839a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:59.823871 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09319d-ac00-49c1-b5b9-eb880c49839a admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:59.824183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09319d-ac00-49c1-b5b9-eb880c49839a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:59.824432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09319d-ac00-49c1-b5b9-eb880c49839a admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:59.825066 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09319d-ac00-49c1-b5b9-eb880c49839a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:53:59.825387 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09319d-ac00-49c1-b5b9-eb880c49839a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:59.826972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09319d-ac00-49c1-b5b9-eb880c49839a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:59.840358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1184/2372] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:53:59.844947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a2f44-bc58-4be0-aa5f-784d253d7e22 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:59.854527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a2f44-bc58-4be0-aa5f-784d253d7e22 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:59.860291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc925f25-9655-4666-918c-42a47ea231ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:59.860922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc925f25-9655-4666-918c-42a47ea231ec admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:59.861197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc925f25-9655-4666-918c-42a47ea231ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:59.861461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc925f25-9655-4666-918c-42a47ea231ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:59.862092 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc925f25-9655-4666-918c-42a47ea231ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:59.864026 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc925f25-9655-4666-918c-42a47ea231ec admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:59.893570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1189/2373] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:53:59 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:53:59.898297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09319d-ac00-49c1-b5b9-eb880c49839a admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:59.908785 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09319d-ac00-49c1-b5b9-eb880c49839a admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:59.954147 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:59.968666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc692568-cd13-4b6c-bf53-805fbbe166d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:59.969324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc692568-cd13-4b6c-bf53-805fbbe166d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:59.969758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc692568-cd13-4b6c-bf53-805fbbe166d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:59.970037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc692568-cd13-4b6c-bf53-805fbbe166d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:59.970727 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc692568-cd13-4b6c-bf53-805fbbe166d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:59.972926 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc692568-cd13-4b6c-bf53-805fbbe166d2 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:59.996460 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1185/2374] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:53:59 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:54:00.000742 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc925f25-9655-4666-918c-42a47ea231ec admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:00.011608 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc925f25-9655-4666-918c-42a47ea231ec admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:00.016255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d501b-6d0c-4ef2-90b2-6b7dd899635b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:00.016866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d501b-6d0c-4ef2-90b2-6b7dd899635b admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:00.017130 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d501b-6d0c-4ef2-90b2-6b7dd899635b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:00.017415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d501b-6d0c-4ef2-90b2-6b7dd899635b admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:00.018289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d501b-6d0c-4ef2-90b2-6b7dd899635b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:00.019685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d501b-6d0c-4ef2-90b2-6b7dd899635b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:00.030821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1190/2375] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:54:00.035178 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc692568-cd13-4b6c-bf53-805fbbe166d2 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:00.046335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc692568-cd13-4b6c-bf53-805fbbe166d2 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:00.052998 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7930dc3-9609-4a8b-8ed0-c401c1e844ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:00.053728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7930dc3-9609-4a8b-8ed0-c401c1e844ef admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:00.054010 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7930dc3-9609-4a8b-8ed0-c401c1e844ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:00.054279 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7930dc3-9609-4a8b-8ed0-c401c1e844ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:00.054907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7930dc3-9609-4a8b-8ed0-c401c1e844ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:00.056592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7930dc3-9609-4a8b-8ed0-c401c1e844ef admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:00.068455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1186/2376] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:54:00.073298 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d501b-6d0c-4ef2-90b2-6b7dd899635b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:00.085254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d501b-6d0c-4ef2-90b2-6b7dd899635b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:00.090463 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbfd7eb-52ed-4e40-9409-cb3e96ad611e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:00.091199 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfd7eb-52ed-4e40-9409-cb3e96ad611e admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:00.091424 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfd7eb-52ed-4e40-9409-cb3e96ad611e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:00.091688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfd7eb-52ed-4e40-9409-cb3e96ad611e admin admin] PATH_INFO: `/v3/projects/92dbec1c0fb249e8983721ac64294809/users/b710395cb34e405c8443ec562a7bc6cc/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:00.108659 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbfd7eb-52ed-4e40-9409-cb3e96ad611e admin admin] RBAC: Authorizing `identity:create_grant(project_id=92dbec1c0fb249e8983721ac64294809, user_id=b710395cb34e405c8443ec562a7bc6cc, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:00.111311 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbfd7eb-52ed-4e40-9409-cb3e96ad611e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:00.128186 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1191/2377] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:54:00 2022] PUT /identity/v3/projects/92dbec1c0fb249e8983721ac64294809/users/b710395cb34e405c8443ec562a7bc6cc/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:54:00.139387 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c8b33-7f94-4745-b8f4-353cded1c8ea None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:00.139663 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c8b33-7f94-4745-b8f4-353cded1c8ea None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:00.139938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c8b33-7f94-4745-b8f4-353cded1c8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:00.171494 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-820c8b33-7f94-4745-b8f4-353cded1c8ea None None] MFA Rules not processed for user `b710395cb34e405c8443ec562a7bc6cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:00.172763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-820c8b33-7f94-4745-b8f4-353cded1c8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:00.239127 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1187/2378] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:54:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:00.254417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbfd7eb-52ed-4e40-9409-cb3e96ad611e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:00.256523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ccbfd7eb-52ed-4e40-9409-cb3e96ad611e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:00.324503 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbfd7eb-52ed-4e40-9409-cb3e96ad611e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:00.328494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc580a-1e05-469c-b2d0-bf42369a649d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:00.329113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc580a-1e05-469c-b2d0-bf42369a649d service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:00.329357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc580a-1e05-469c-b2d0-bf42369a649d service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:00.329656 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc580a-1e05-469c-b2d0-bf42369a649d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:00.338542 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc580a-1e05-469c-b2d0-bf42369a649d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:00.339988 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc580a-1e05-469c-b2d0-bf42369a649d service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:00.349526 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1192/2379] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:54:00.370725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c8b33-7f94-4745-b8f4-353cded1c8ea None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:00.449540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c8b33-7f94-4745-b8f4-353cded1c8ea None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:00.454930 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4d473-0eee-4270-a3f7-2605ec5e3cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:00.455563 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4d473-0eee-4270-a3f7-2605ec5e3cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:00.455842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4d473-0eee-4270-a3f7-2605ec5e3cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:00.456142 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4d473-0eee-4270-a3f7-2605ec5e3cae admin admin] PATH_INFO: `/v3/users/b710395cb34e405c8443ec562a7bc6cc` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:00.457408 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4d473-0eee-4270-a3f7-2605ec5e3cae admin admin] RBAC: Authorizing `identity:delete_user(user_id=b710395cb34e405c8443ec562a7bc6cc)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:00.459357 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4d473-0eee-4270-a3f7-2605ec5e3cae admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:00.512545 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8bb4d473-0eee-4270-a3f7-2605ec5e3cae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b710395cb34e405c8443ec562a7bc6cc'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:00.513202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8bb4d473-0eee-4270-a3f7-2605ec5e3cae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b710395cb34e405c8443ec562a7bc6cc'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:00.513801 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8bb4d473-0eee-4270-a3f7-2605ec5e3cae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b710395cb34e405c8443ec562a7bc6cc'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:00.523603 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8bb4d473-0eee-4270-a3f7-2605ec5e3cae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b710395cb34e405c8443ec562a7bc6cc'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:00.525799 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:00.544051 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-8bb4d473-0eee-4270-a3f7-2605ec5e3cae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b710395cb34e405c8443ec562a7bc6cc'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:00.557762 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1188/2380] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:54:00 2022] DELETE /identity/v3/users/b710395cb34e405c8443ec562a7bc6cc => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:54:00.562071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d47677-0943-407d-b1fe-4a5956ff300d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:00.562354 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d47677-0943-407d-b1fe-4a5956ff300d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:00.562625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d47677-0943-407d-b1fe-4a5956ff300d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:00.576654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:00.583273 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-55d47677-0943-407d-b1fe-4a5956ff300d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:00.585945 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-55d47677-0943-407d-b1fe-4a5956ff300d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:00.660894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1193/2381] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:00.669417 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4d473-0eee-4270-a3f7-2605ec5e3cae admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:00.670890 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-8bb4d473-0eee-4270-a3f7-2605ec5e3cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:00.730685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4d473-0eee-4270-a3f7-2605ec5e3cae admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:00.734539 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935cd48-1ad2-480f-b847-494a99eff4f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:00.735164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935cd48-1ad2-480f-b847-494a99eff4f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:00.735441 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935cd48-1ad2-480f-b847-494a99eff4f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:00.735711 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935cd48-1ad2-480f-b847-494a99eff4f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:00.745580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935cd48-1ad2-480f-b847-494a99eff4f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:00.747021 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935cd48-1ad2-480f-b847-494a99eff4f4 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:00.757446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1189/2382] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:01.069385 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d47677-0943-407d-b1fe-4a5956ff300d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:01.071601 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-55d47677-0943-407d-b1fe-4a5956ff300d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:01.088144 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d47677-0943-407d-b1fe-4a5956ff300d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:01.093664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0abcc4-ff84-4af1-a233-fb859f3261ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:01.094372 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0abcc4-ff84-4af1-a233-fb859f3261ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:01.094579 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0abcc4-ff84-4af1-a233-fb859f3261ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:01.094845 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0abcc4-ff84-4af1-a233-fb859f3261ea admin admin] PATH_INFO: `/v3/projects/92dbec1c0fb249e8983721ac64294809` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:01.096124 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0abcc4-ff84-4af1-a233-fb859f3261ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=92dbec1c0fb249e8983721ac64294809)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:01.098042 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0abcc4-ff84-4af1-a233-fb859f3261ea admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:01.147039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9f0abcc4-ff84-4af1-a233-fb859f3261ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92dbec1c0fb249e8983721ac64294809', 'request_id': 'req-9f0abcc4-ff84-4af1-a233-fb859f3261ea'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:01.156242 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9f0abcc4-ff84-4af1-a233-fb859f3261ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92dbec1c0fb249e8983721ac64294809', 'request_id': 'req-9f0abcc4-ff84-4af1-a233-fb859f3261ea'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:01.162648 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9f0abcc4-ff84-4af1-a233-fb859f3261ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92dbec1c0fb249e8983721ac64294809', 'request_id': 'req-9f0abcc4-ff84-4af1-a233-fb859f3261ea'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:01.176408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9f0abcc4-ff84-4af1-a233-fb859f3261ea admin admin] The token cache is being invalidate because project 92dbec1c0fb249e8983721ac64294809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:54:01.176692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9f0abcc4-ff84-4af1-a233-fb859f3261ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:01.177849 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9f0abcc4-ff84-4af1-a233-fb859f3261ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:01.180006 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1194/2383] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:54:01 2022] DELETE /identity/v3/projects/92dbec1c0fb249e8983721ac64294809 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:54:01.186347 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d066a9-7756-47d5-aba6-24575144c6f9 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:01.186657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d066a9-7756-47d5-aba6-24575144c6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:01.186922 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d066a9-7756-47d5-aba6-24575144c6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:01.210641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-b9d066a9-7756-47d5-aba6-24575144c6f9 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:01.211894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b9d066a9-7756-47d5-aba6-24575144c6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:01.275493 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1190/2384] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:01.346932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bece41-29d5-4863-8412-19950e3e5407 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:01.348514 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bece41-29d5-4863-8412-19950e3e5407 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:01.349011 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bece41-29d5-4863-8412-19950e3e5407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:01.375371 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-35bece41-29d5-4863-8412-19950e3e5407 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:01.376632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-35bece41-29d5-4863-8412-19950e3e5407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:01.384992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1195/2385] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:01.390340 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d066a9-7756-47d5-aba6-24575144c6f9 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:01.403450 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d066a9-7756-47d5-aba6-24575144c6f9 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:01.408935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f03d23d-44a3-4b53-be80-9a93706aa39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:01.409555 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03d23d-44a3-4b53-be80-9a93706aa39a admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:01.409830 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03d23d-44a3-4b53-be80-9a93706aa39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:01.410095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03d23d-44a3-4b53-be80-9a93706aa39a admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:01.410750 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f03d23d-44a3-4b53-be80-9a93706aa39a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:54:01.411045 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f03d23d-44a3-4b53-be80-9a93706aa39a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:01.412719 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f03d23d-44a3-4b53-be80-9a93706aa39a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:01.426359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1191/2386] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:54:01.433073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bece41-29d5-4863-8412-19950e3e5407 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:01.443179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bece41-29d5-4863-8412-19950e3e5407 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:01.449079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c0550-d4d4-406b-ac8a-fa0dd7c7828e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:01.449695 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c0550-d4d4-406b-ac8a-fa0dd7c7828e admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:01.449971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c0550-d4d4-406b-ac8a-fa0dd7c7828e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:01.450243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c0550-d4d4-406b-ac8a-fa0dd7c7828e admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:01.450874 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c0550-d4d4-406b-ac8a-fa0dd7c7828e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:01.452851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c0550-d4d4-406b-ac8a-fa0dd7c7828e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:01.483415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1196/2387] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:54:01 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:54:01.488086 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f03d23d-44a3-4b53-be80-9a93706aa39a admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:01.499779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f03d23d-44a3-4b53-be80-9a93706aa39a admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:01.569550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7547161-fd7e-47e3-b19e-79fdaa01881d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:01.570497 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7547161-fd7e-47e3-b19e-79fdaa01881d admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:01.571017 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7547161-fd7e-47e3-b19e-79fdaa01881d admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:01.571413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7547161-fd7e-47e3-b19e-79fdaa01881d admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:01.572295 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7547161-fd7e-47e3-b19e-79fdaa01881d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:01.574969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7547161-fd7e-47e3-b19e-79fdaa01881d admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:01.589450 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:01.602920 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1192/2388] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:54:01 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:54:01.607157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c0550-d4d4-406b-ac8a-fa0dd7c7828e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:01.618532 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c0550-d4d4-406b-ac8a-fa0dd7c7828e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:01.623460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bf69a-0fff-4cd4-8bbb-3e4402ccc2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:01.624103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bf69a-0fff-4cd4-8bbb-3e4402ccc2f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:01.624381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bf69a-0fff-4cd4-8bbb-3e4402ccc2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:01.624648 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bf69a-0fff-4cd4-8bbb-3e4402ccc2f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:01.625305 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bf69a-0fff-4cd4-8bbb-3e4402ccc2f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:01.626911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bf69a-0fff-4cd4-8bbb-3e4402ccc2f4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:01.638815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1197/2389] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:54:01.643929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7547161-fd7e-47e3-b19e-79fdaa01881d admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:01.655541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7547161-fd7e-47e3-b19e-79fdaa01881d admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:01.661573 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c9c74-01e1-4f36-b462-c014b87f31de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:01.662196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c9c74-01e1-4f36-b462-c014b87f31de admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:01.662471 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c9c74-01e1-4f36-b462-c014b87f31de admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:01.662735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c9c74-01e1-4f36-b462-c014b87f31de admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:01.663373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c9c74-01e1-4f36-b462-c014b87f31de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:01.665145 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c9c74-01e1-4f36-b462-c014b87f31de admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:01.676992 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1193/2390] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:54:01.682026 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bf69a-0fff-4cd4-8bbb-3e4402ccc2f4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:01.692570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bf69a-0fff-4cd4-8bbb-3e4402ccc2f4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:01.698501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a8bd9-95d4-4c08-b33b-4e935d2a226f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:01.698734 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a8bd9-95d4-4c08-b33b-4e935d2a226f admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:01.698931 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a8bd9-95d4-4c08-b33b-4e935d2a226f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:01.699228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a8bd9-95d4-4c08-b33b-4e935d2a226f admin admin] PATH_INFO: `/v3/projects/90721ed18d554cecbb40eb8d72b69821/users/71f8b0d50b2942a1bbb6c7164b3307a2/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:01.716718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a8bd9-95d4-4c08-b33b-4e935d2a226f admin admin] RBAC: Authorizing `identity:create_grant(project_id=90721ed18d554cecbb40eb8d72b69821, user_id=71f8b0d50b2942a1bbb6c7164b3307a2, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:01.719561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a8bd9-95d4-4c08-b33b-4e935d2a226f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:01.734185 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1198/2391] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:54:01 2022] PUT /identity/v3/projects/90721ed18d554cecbb40eb8d72b69821/users/71f8b0d50b2942a1bbb6c7164b3307a2/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:54:01.745043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64885ca4-cb38-4eb7-85f4-058eab91b0c1 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:01.745329 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64885ca4-cb38-4eb7-85f4-058eab91b0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:01.745700 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64885ca4-cb38-4eb7-85f4-058eab91b0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:01.777533 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-64885ca4-cb38-4eb7-85f4-058eab91b0c1 None None] MFA Rules not processed for user `71f8b0d50b2942a1bbb6c7164b3307a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:01.778770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-64885ca4-cb38-4eb7-85f4-058eab91b0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:01.848752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1194/2392] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:54:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:01.861603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a8bd9-95d4-4c08-b33b-4e935d2a226f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:01.863728 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e22a8bd9-95d4-4c08-b33b-4e935d2a226f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:01.897658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:01.931848 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a8bd9-95d4-4c08-b33b-4e935d2a226f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:01.935633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a33c85-5df5-4da7-8b26-433e1f13280e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:01.936278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a33c85-5df5-4da7-8b26-433e1f13280e service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:01.936552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a33c85-5df5-4da7-8b26-433e1f13280e service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:01.936819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a33c85-5df5-4da7-8b26-433e1f13280e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:01.946189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a33c85-5df5-4da7-8b26-433e1f13280e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:01.947658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a33c85-5df5-4da7-8b26-433e1f13280e service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:01.957342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1199/2393] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:05.113556 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64885ca4-cb38-4eb7-85f4-058eab91b0c1 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:05.182087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64885ca4-cb38-4eb7-85f4-058eab91b0c1 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:05.187083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddac731-5d36-405d-a1c2-ef36892c48e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:05.187709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddac731-5d36-405d-a1c2-ef36892c48e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:05.187981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddac731-5d36-405d-a1c2-ef36892c48e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:05.188277 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddac731-5d36-405d-a1c2-ef36892c48e0 admin admin] PATH_INFO: `/v3/users/71f8b0d50b2942a1bbb6c7164b3307a2` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:05.189436 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddac731-5d36-405d-a1c2-ef36892c48e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71f8b0d50b2942a1bbb6c7164b3307a2)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:05.191334 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddac731-5d36-405d-a1c2-ef36892c48e0 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:05.243387 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-3ddac731-5d36-405d-a1c2-ef36892c48e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71f8b0d50b2942a1bbb6c7164b3307a2'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:05.244037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-3ddac731-5d36-405d-a1c2-ef36892c48e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71f8b0d50b2942a1bbb6c7164b3307a2'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:05.244647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-3ddac731-5d36-405d-a1c2-ef36892c48e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71f8b0d50b2942a1bbb6c7164b3307a2'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:05.252944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-3ddac731-5d36-405d-a1c2-ef36892c48e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71f8b0d50b2942a1bbb6c7164b3307a2'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:05.265927 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-3ddac731-5d36-405d-a1c2-ef36892c48e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71f8b0d50b2942a1bbb6c7164b3307a2'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:05.272054 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:05.283413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1195/2394] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:54:05 2022] DELETE /identity/v3/users/71f8b0d50b2942a1bbb6c7164b3307a2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:54:05.289511 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636769bd-2cdc-49c8-9371-e7ec0e9a1689 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:05.290079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636769bd-2cdc-49c8-9371-e7ec0e9a1689 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:05.290947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636769bd-2cdc-49c8-9371-e7ec0e9a1689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:05.311824 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-636769bd-2cdc-49c8-9371-e7ec0e9a1689 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:05.313062 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-636769bd-2cdc-49c8-9371-e7ec0e9a1689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:05.371341 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1200/2395] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:54:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:05.381628 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddac731-5d36-405d-a1c2-ef36892c48e0 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:05.384039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-3ddac731-5d36-405d-a1c2-ef36892c48e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:05.448107 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddac731-5d36-405d-a1c2-ef36892c48e0 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:05.451857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96523db9-af26-480e-b5d7-d34f63a36fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:05.452513 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96523db9-af26-480e-b5d7-d34f63a36fbd service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:05.452788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96523db9-af26-480e-b5d7-d34f63a36fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:05.453056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96523db9-af26-480e-b5d7-d34f63a36fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:05.463226 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96523db9-af26-480e-b5d7-d34f63a36fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:05.464762 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96523db9-af26-480e-b5d7-d34f63a36fbd service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:05.474793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1196/2396] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:05.485487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636769bd-2cdc-49c8-9371-e7ec0e9a1689 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:05.495676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636769bd-2cdc-49c8-9371-e7ec0e9a1689 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:05.499801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635713b-3b19-46e6-86b2-19e504a456d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:05.500449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635713b-3b19-46e6-86b2-19e504a456d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:05.500728 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635713b-3b19-46e6-86b2-19e504a456d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:05.500982 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635713b-3b19-46e6-86b2-19e504a456d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:05.502758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9635713b-3b19-46e6-86b2-19e504a456d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:05.586038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635713b-3b19-46e6-86b2-19e504a456d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:05.587520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635713b-3b19-46e6-86b2-19e504a456d9 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:05.599547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1201/2397] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:06.123348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96523db9-af26-480e-b5d7-d34f63a36fbd service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:06.124978 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-96523db9-af26-480e-b5d7-d34f63a36fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:06.139074 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96523db9-af26-480e-b5d7-d34f63a36fbd service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:06.145157 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72be78-b819-4b5a-b292-eaed03f8972c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:06.145996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72be78-b819-4b5a-b292-eaed03f8972c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:06.146366 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72be78-b819-4b5a-b292-eaed03f8972c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:06.146734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72be78-b819-4b5a-b292-eaed03f8972c admin admin] PATH_INFO: `/v3/projects/90721ed18d554cecbb40eb8d72b69821` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:06.148314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb72be78-b819-4b5a-b292-eaed03f8972c admin admin] RBAC: Authorizing `identity:delete_project(project_id=90721ed18d554cecbb40eb8d72b69821)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:06.150578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb72be78-b819-4b5a-b292-eaed03f8972c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:06.208581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bb72be78-b819-4b5a-b292-eaed03f8972c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90721ed18d554cecbb40eb8d72b69821', 'request_id': 'req-bb72be78-b819-4b5a-b292-eaed03f8972c'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:06.214402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bb72be78-b819-4b5a-b292-eaed03f8972c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90721ed18d554cecbb40eb8d72b69821', 'request_id': 'req-bb72be78-b819-4b5a-b292-eaed03f8972c'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:06.220844 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bb72be78-b819-4b5a-b292-eaed03f8972c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90721ed18d554cecbb40eb8d72b69821', 'request_id': 'req-bb72be78-b819-4b5a-b292-eaed03f8972c'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:06.234965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bb72be78-b819-4b5a-b292-eaed03f8972c admin admin] The token cache is being invalidate because project 90721ed18d554cecbb40eb8d72b69821 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:54:06.235259 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bb72be78-b819-4b5a-b292-eaed03f8972c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:06.235896 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bb72be78-b819-4b5a-b292-eaed03f8972c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:54:06.237630 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1197/2398] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:54:06 2022] DELETE /identity/v3/projects/90721ed18d554cecbb40eb8d72b69821 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:54:06.242607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa83058-3c6e-434d-a2b4-87c15df7226a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:06.242894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa83058-3c6e-434d-a2b4-87c15df7226a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:06.243168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa83058-3c6e-434d-a2b4-87c15df7226a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:06.263630 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-dfa83058-3c6e-434d-a2b4-87c15df7226a None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:06.264917 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-dfa83058-3c6e-434d-a2b4-87c15df7226a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:06.314833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:06.327581 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1202/2399] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:06.335268 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdad19-6de9-494d-9e6a-34d22c08c640 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:06.335732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdad19-6de9-494d-9e6a-34d22c08c640 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:06.336022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdad19-6de9-494d-9e6a-34d22c08c640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:06.357030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-d0fdad19-6de9-494d-9e6a-34d22c08c640 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:06.358245 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d0fdad19-6de9-494d-9e6a-34d22c08c640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:06.366932 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1198/2400] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:06.371927 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa83058-3c6e-434d-a2b4-87c15df7226a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:06.385553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa83058-3c6e-434d-a2b4-87c15df7226a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:06.396350 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198d5ff-80e0-40cc-99ca-9aca2c69350c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:06.397048 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198d5ff-80e0-40cc-99ca-9aca2c69350c admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:06.397346 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198d5ff-80e0-40cc-99ca-9aca2c69350c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:06.397612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198d5ff-80e0-40cc-99ca-9aca2c69350c admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:06.398368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198d5ff-80e0-40cc-99ca-9aca2c69350c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:54:06.398635 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198d5ff-80e0-40cc-99ca-9aca2c69350c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:06.400446 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198d5ff-80e0-40cc-99ca-9aca2c69350c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:06.417619 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1203/2401] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:54:06.422907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdad19-6de9-494d-9e6a-34d22c08c640 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:06.433489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdad19-6de9-494d-9e6a-34d22c08c640 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:06.438383 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593ecc2-73c3-44b2-a5b7-72c4fee4dc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:06.438970 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593ecc2-73c3-44b2-a5b7-72c4fee4dc7a admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:06.439233 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593ecc2-73c3-44b2-a5b7-72c4fee4dc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:06.439668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593ecc2-73c3-44b2-a5b7-72c4fee4dc7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:06.440621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593ecc2-73c3-44b2-a5b7-72c4fee4dc7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:06.442338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593ecc2-73c3-44b2-a5b7-72c4fee4dc7a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:06.472125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1199/2402] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:54:06 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:54:06.476852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198d5ff-80e0-40cc-99ca-9aca2c69350c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:06.487543 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198d5ff-80e0-40cc-99ca-9aca2c69350c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:06.559755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7855ec0-0aa0-46fa-882c-e27a5e4ed2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:06.560403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7855ec0-0aa0-46fa-882c-e27a5e4ed2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:06.560684 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7855ec0-0aa0-46fa-882c-e27a5e4ed2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:06.560951 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7855ec0-0aa0-46fa-882c-e27a5e4ed2b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:06.561600 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7855ec0-0aa0-46fa-882c-e27a5e4ed2b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:06.563533 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7855ec0-0aa0-46fa-882c-e27a5e4ed2b6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:06.587752 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1204/2403] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:54:06 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:54:06.592171 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593ecc2-73c3-44b2-a5b7-72c4fee4dc7a admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:06.602777 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593ecc2-73c3-44b2-a5b7-72c4fee4dc7a admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:06.607731 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd68ad-3c28-4059-8acf-2187a0555ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:06.608365 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd68ad-3c28-4059-8acf-2187a0555ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:06.608647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd68ad-3c28-4059-8acf-2187a0555ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:06.608915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd68ad-3c28-4059-8acf-2187a0555ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:06.609562 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd68ad-3c28-4059-8acf-2187a0555ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:06.611157 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd68ad-3c28-4059-8acf-2187a0555ba9 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:06.621497 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1200/2404] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:54:06.625698 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7855ec0-0aa0-46fa-882c-e27a5e4ed2b6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:06.636860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7855ec0-0aa0-46fa-882c-e27a5e4ed2b6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:06.642281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4223d13-fa42-4fe7-8679-f12de22b82c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:06.642920 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4223d13-fa42-4fe7-8679-f12de22b82c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:06.643196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4223d13-fa42-4fe7-8679-f12de22b82c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:06.643460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4223d13-fa42-4fe7-8679-f12de22b82c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:06.644139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4223d13-fa42-4fe7-8679-f12de22b82c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:06.645756 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4223d13-fa42-4fe7-8679-f12de22b82c0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:06.657035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1205/2405] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:54:06.661543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd68ad-3c28-4059-8acf-2187a0555ba9 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:06.671980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd68ad-3c28-4059-8acf-2187a0555ba9 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:06.677594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23db3c-ec69-4239-be65-0164f9c7ec6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:06.678291 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23db3c-ec69-4239-be65-0164f9c7ec6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:06.678567 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23db3c-ec69-4239-be65-0164f9c7ec6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:06.678865 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23db3c-ec69-4239-be65-0164f9c7ec6c admin admin] PATH_INFO: `/v3/projects/ac849c366ee54767bac65b18976c7085/users/a20233e0c38e492bad530401f388f19a/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:06.694000 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23db3c-ec69-4239-be65-0164f9c7ec6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac849c366ee54767bac65b18976c7085, user_id=a20233e0c38e492bad530401f388f19a, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:06.696743 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23db3c-ec69-4239-be65-0164f9c7ec6c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:06.712099 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1201/2406] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:54:06 2022] PUT /identity/v3/projects/ac849c366ee54767bac65b18976c7085/users/a20233e0c38e492bad530401f388f19a/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:54:06.717442 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a76ef9-fba7-4569-ae25-9b306ab5ead7 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:06.717726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a76ef9-fba7-4569-ae25-9b306ab5ead7 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:06.717998 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a76ef9-fba7-4569-ae25-9b306ab5ead7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:06.737785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a0a76ef9-fba7-4569-ae25-9b306ab5ead7 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:06.739026 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a0a76ef9-fba7-4569-ae25-9b306ab5ead7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:06.809507 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1206/2407] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:06.822917 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23db3c-ec69-4239-be65-0164f9c7ec6c admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:06.825973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4e23db3c-ec69-4239-be65-0164f9c7ec6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:06.844522 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:06.901024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23db3c-ec69-4239-be65-0164f9c7ec6c admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:06.904999 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7769a-577f-4910-a152-80d7112b81dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:06.905609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7769a-577f-4910-a152-80d7112b81dc service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:06.905882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7769a-577f-4910-a152-80d7112b81dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:06.906144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7769a-577f-4910-a152-80d7112b81dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:06.915397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7769a-577f-4910-a152-80d7112b81dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:06.916868 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7769a-577f-4910-a152-80d7112b81dc service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:06.926251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1202/2408] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:07.629601 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b295ad-b46b-4592-81f9-59349fdf703d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:07.630678 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b295ad-b46b-4592-81f9-59349fdf703d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:07.630959 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b295ad-b46b-4592-81f9-59349fdf703d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:07.664453 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-93b295ad-b46b-4592-81f9-59349fdf703d None None] MFA Rules not processed for user `a20233e0c38e492bad530401f388f19a`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:07.666010 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-93b295ad-b46b-4592-81f9-59349fdf703d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:07.720828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:07.741127 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1207/2409] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:54:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:07.753418 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7769a-577f-4910-a152-80d7112b81dc service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:07.754964 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-64a7769a-577f-4910-a152-80d7112b81dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:07.817525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7769a-577f-4910-a152-80d7112b81dc service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:07.822032 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d1d53-d06a-45e0-84af-76259e260ae2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:07.822589 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d1d53-d06a-45e0-84af-76259e260ae2 service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:07.822863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d1d53-d06a-45e0-84af-76259e260ae2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:07.823129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d1d53-d06a-45e0-84af-76259e260ae2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:07.832536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d1d53-d06a-45e0-84af-76259e260ae2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:07.834056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d1d53-d06a-45e0-84af-76259e260ae2 service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:07.843543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1203/2410] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:08.351526 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b295ad-b46b-4592-81f9-59349fdf703d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:08.353723 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-93b295ad-b46b-4592-81f9-59349fdf703d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:08.371774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b295ad-b46b-4592-81f9-59349fdf703d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:08.376187 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17736e79-9ba7-4abd-bd43-28d5cbef2763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:08.376815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17736e79-9ba7-4abd-bd43-28d5cbef2763 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:08.377089 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17736e79-9ba7-4abd-bd43-28d5cbef2763 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:08.377356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17736e79-9ba7-4abd-bd43-28d5cbef2763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:08.379542 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-17736e79-9ba7-4abd-bd43-28d5cbef2763 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:08.456039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17736e79-9ba7-4abd-bd43-28d5cbef2763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:08.457530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17736e79-9ba7-4abd-bd43-28d5cbef2763 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:08.469180 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1208/2411] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:09.145789 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3198-9da5-4583-824f-c31f08917152 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:09.146274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3198-9da5-4583-824f-c31f08917152 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:09.146399 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3198-9da5-4583-824f-c31f08917152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:09.166837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-69bd3198-9da5-4583-824f-c31f08917152 None None] MFA Rules not processed for user `8fc940594a1848aa9644b645049a1bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:09.168430 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-69bd3198-9da5-4583-824f-c31f08917152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:09.227934 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a8a24-6028-4c83-bd91-4558cd9ca454 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:09.228529 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a8a24-6028-4c83-bd91-4558cd9ca454 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:09.228529 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a8a24-6028-4c83-bd91-4558cd9ca454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:09.230285 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1204/2412] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Aug 30 14:54:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:09.248481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-614a8a24-6028-4c83-bd91-4558cd9ca454 None None] MFA Rules not processed for user `a20233e0c38e492bad530401f388f19a`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:09.249591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-614a8a24-6028-4c83-bd91-4558cd9ca454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:09.257243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1209/2413] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:54:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:54:09.282168 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd3198-9da5-4583-824f-c31f08917152 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:09.283887 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-69bd3198-9da5-4583-824f-c31f08917152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:09.358530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd3198-9da5-4583-824f-c31f08917152 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:09.363685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e9f5c-f03c-46d9-b953-87494425cf87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d98f5adfaf410580481417e7ba1638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:09.364696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9f5c-f03c-46d9-b953-87494425cf87 service swift] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:09.365127 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9f5c-f03c-46d9-b953-87494425cf87 service swift] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:09.365393 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9f5c-f03c-46d9-b953-87494425cf87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:09.371375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:09.375535 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e9f5c-f03c-46d9-b953-87494425cf87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:09.377397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e9f5c-f03c-46d9-b953-87494425cf87 service swift] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:09.386727 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1205/2414] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Aug 30 14:54:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:54:16.445793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a8a24-6028-4c83-bd91-4558cd9ca454 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:16.448162 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-614a8a24-6028-4c83-bd91-4558cd9ca454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:16.470447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a8a24-6028-4c83-bd91-4558cd9ca454 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:16.481540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd350fd-357d-45f0-a29a-1d97efca5b61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:16.482580 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd350fd-357d-45f0-a29a-1d97efca5b61 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:16.482999 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd350fd-357d-45f0-a29a-1d97efca5b61 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:16.483569 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd350fd-357d-45f0-a29a-1d97efca5b61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:16.485377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3bd350fd-357d-45f0-a29a-1d97efca5b61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:16.590661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd350fd-357d-45f0-a29a-1d97efca5b61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:16.592556 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd350fd-357d-45f0-a29a-1d97efca5b61 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:16.614961 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1210/2415] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:19.382408 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:26.611775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=82 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:27.518858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1be12-bcfc-4f75-8a73-09d0436de6ad None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:27.519459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1be12-bcfc-4f75-8a73-09d0436de6ad None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:27.519459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1be12-bcfc-4f75-8a73-09d0436de6ad None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:27.520813 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1206/2416] 10.222.0.25 () {54 vars in 921 bytes} [Tue Aug 30 14:54:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:54:27.527011 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c762f9-8299-402a-8271-719292522b7d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:27.527374 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c762f9-8299-402a-8271-719292522b7d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:27.527660 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c762f9-8299-402a-8271-719292522b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:27.570340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-86c762f9-8299-402a-8271-719292522b7d None None] MFA Rules not processed for user `cc95bbdafb0c4f34ad0662bc2e3ba4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:27.571691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-86c762f9-8299-402a-8271-719292522b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:27.583678 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1211/2417] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Aug 30 14:54:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:27.599053 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1be12-bcfc-4f75-8a73-09d0436de6ad None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:27.600891 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-deb1be12-bcfc-4f75-8a73-09d0436de6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:27.674813 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1be12-bcfc-4f75-8a73-09d0436de6ad None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:27.679372 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5028f24-ecc8-456c-b923-b7239c180068 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69de9368993f402cb6325e5022e064d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:27.680108 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5028f24-ecc8-456c-b923-b7239c180068 service glance] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:27.680450 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5028f24-ecc8-456c-b923-b7239c180068 service glance] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:27.680781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5028f24-ecc8-456c-b923-b7239c180068 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:27.693136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5028f24-ecc8-456c-b923-b7239c180068 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:27.694589 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5028f24-ecc8-456c-b923-b7239c180068 service glance] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:27.707004 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1207/2418] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:54:27.779674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbeb076-dec3-4ef6-8495-9ec79b5a8d08 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:27.780322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbeb076-dec3-4ef6-8495-9ec79b5a8d08 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:27.780322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbeb076-dec3-4ef6-8495-9ec79b5a8d08 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:27.781662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1212/2419] 10.222.0.25 () {54 vars in 921 bytes} [Tue Aug 30 14:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:54:27.786424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8cb64-a84c-457f-aa65-8be10bd83f80 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:27.786728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8cb64-a84c-457f-aa65-8be10bd83f80 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:27.787022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8cb64-a84c-457f-aa65-8be10bd83f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:27.827854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-09b8cb64-a84c-457f-aa65-8be10bd83f80 None None] MFA Rules not processed for user `932fce741c5a4deba79a310a14e6bd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:27.829113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-09b8cb64-a84c-457f-aa65-8be10bd83f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:27.836062 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1208/2420] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Aug 30 14:54:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:27.848064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbeb076-dec3-4ef6-8495-9ec79b5a8d08 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:27.859431 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbeb076-dec3-4ef6-8495-9ec79b5a8d08 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:27.863141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9a9aa-7c32-45ac-a8b0-f7b9b7f14393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:27.863742 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9a9aa-7c32-45ac-a8b0-f7b9b7f14393 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:27.864028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9a9aa-7c32-45ac-a8b0-f7b9b7f14393 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:27.864335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9a9aa-7c32-45ac-a8b0-f7b9b7f14393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:27.872220 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9a9aa-7c32-45ac-a8b0-f7b9b7f14393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:27.873673 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9a9aa-7c32-45ac-a8b0-f7b9b7f14393 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:27.881780 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1213/2421] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:27.972941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a55f7-7cb2-454a-81de-aa888b6031f1 None None] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:27.973275 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a55f7-7cb2-454a-81de-aa888b6031f1 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:27.973573 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a55f7-7cb2-454a-81de-aa888b6031f1 None None] PATH_INFO: `/` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:27.974951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1209/2422] 10.222.0.25 () {54 vars in 921 bytes} [Tue Aug 30 14:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:54:27.984939 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cfa27d-0317-4ff7-a5b5-0718398e6244 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:27.985194 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cfa27d-0317-4ff7-a5b5-0718398e6244 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:27.985491 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cfa27d-0317-4ff7-a5b5-0718398e6244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:28.008761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-25cfa27d-0317-4ff7-a5b5-0718398e6244 None None] MFA Rules not processed for user `cc95bbdafb0c4f34ad0662bc2e3ba4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:28.009979 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-25cfa27d-0317-4ff7-a5b5-0718398e6244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:28.021942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1214/2423] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Aug 30 14:54:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:28.034910 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a55f7-7cb2-454a-81de-aa888b6031f1 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:28.047499 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a55f7-7cb2-454a-81de-aa888b6031f1 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:28.051256 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949ce81-9bdc-4720-a2d1-9a487bfe8112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:28.052233 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949ce81-9bdc-4720-a2d1-9a487bfe8112 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:28.052511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949ce81-9bdc-4720-a2d1-9a487bfe8112 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:28.052835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949ce81-9bdc-4720-a2d1-9a487bfe8112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:28.063612 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949ce81-9bdc-4720-a2d1-9a487bfe8112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:28.065062 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949ce81-9bdc-4720-a2d1-9a487bfe8112 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:28.076851 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1210/2424] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:28.111905 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87484efa-f92b-4db8-a8d0-5f4459d371ee None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:28.113176 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87484efa-f92b-4db8-a8d0-5f4459d371ee None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:28.113176 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87484efa-f92b-4db8-a8d0-5f4459d371ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:28.133083 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-87484efa-f92b-4db8-a8d0-5f4459d371ee None None] MFA Rules not processed for user `a20233e0c38e492bad530401f388f19a`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:28.134431 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-87484efa-f92b-4db8-a8d0-5f4459d371ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:28.142686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1215/2425] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:54:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:28.150746 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949ce81-9bdc-4720-a2d1-9a487bfe8112 service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:28.152359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e949ce81-9bdc-4720-a2d1-9a487bfe8112 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:28.236712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949ce81-9bdc-4720-a2d1-9a487bfe8112 service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:28.242022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940eef57-289a-45c2-b912-11cabc823029 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cdd6b1ee9f4e97beb9bb037e446c8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:28.242643 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eef57-289a-45c2-b912-11cabc823029 service placement] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:28.242919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eef57-289a-45c2-b912-11cabc823029 service placement] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:28.243186 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eef57-289a-45c2-b912-11cabc823029 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:28.244926 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-940eef57-289a-45c2-b912-11cabc823029 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:28.257561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940eef57-289a-45c2-b912-11cabc823029 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:28.259166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940eef57-289a-45c2-b912-11cabc823029 service placement] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:28.269517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1211/2426] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:30.709693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae92c0-815e-4699-a822-142c6056e199 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:30.710199 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae92c0-815e-4699-a822-142c6056e199 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:30.710199 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae92c0-815e-4699-a822-142c6056e199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:30.730478 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c6ae92c0-815e-4699-a822-142c6056e199 None None] MFA Rules not processed for user `8fc940594a1848aa9644b645049a1bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:30.731841 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c6ae92c0-815e-4699-a822-142c6056e199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:30.739367 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1216/2427] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Aug 30 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:30.752919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940eef57-289a-45c2-b912-11cabc823029 service placement] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:30.765482 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940eef57-289a-45c2-b912-11cabc823029 service placement] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:30.772130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c747b-222a-4958-995a-64828ff1f334 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d98f5adfaf410580481417e7ba1638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:30.772860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c747b-222a-4958-995a-64828ff1f334 service swift] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:30.773199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c747b-222a-4958-995a-64828ff1f334 service swift] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:30.774180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c747b-222a-4958-995a-64828ff1f334 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:30.783138 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c747b-222a-4958-995a-64828ff1f334 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:30.784861 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c747b-222a-4958-995a-64828ff1f334 service swift] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:30.793658 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1212/2428] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Aug 30 14:54:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:54:40.731230 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=24 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:40.790139 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=79 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:44.333978 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae92c0-815e-4699-a822-142c6056e199 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:44.344604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae92c0-815e-4699-a822-142c6056e199 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:44.349729 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4dca1-3507-49c7-9308-8bc1f73bbb19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:44.350514 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4dca1-3507-49c7-9308-8bc1f73bbb19 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:44.350900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4dca1-3507-49c7-9308-8bc1f73bbb19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:44.351173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4dca1-3507-49c7-9308-8bc1f73bbb19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:44.353393 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-dad4dca1-3507-49c7-9308-8bc1f73bbb19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:44.364224 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4dca1-3507-49c7-9308-8bc1f73bbb19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:44.365700 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4dca1-3507-49c7-9308-8bc1f73bbb19 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:44.373701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1217/2429] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:52.879236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237be649-17da-4ade-a5b5-6c5b2353e261 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:52.879854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237be649-17da-4ade-a5b5-6c5b2353e261 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:52.879854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237be649-17da-4ade-a5b5-6c5b2353e261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:52.941102 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-237be649-17da-4ade-a5b5-6c5b2353e261 None None] MFA Rules not processed for user `8fc940594a1848aa9644b645049a1bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:52.942360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-237be649-17da-4ade-a5b5-6c5b2353e261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:52.960816 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1213/2430] 10.222.0.25 () {58 vars in 967 bytes} [Tue Aug 30 14:54:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:52.975100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4dca1-3507-49c7-9308-8bc1f73bbb19 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:52.999499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4dca1-3507-49c7-9308-8bc1f73bbb19 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:53.005593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb8580-3e24-41fd-8d0b-555dd4fe4cb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d98f5adfaf410580481417e7ba1638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:53.006221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb8580-3e24-41fd-8d0b-555dd4fe4cb9 service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:53.006577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb8580-3e24-41fd-8d0b-555dd4fe4cb9 service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:53.006767 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb8580-3e24-41fd-8d0b-555dd4fe4cb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:53.015931 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb8580-3e24-41fd-8d0b-555dd4fe4cb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:53.017800 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb8580-3e24-41fd-8d0b-555dd4fe4cb9 service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:53.027682 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1218/2431] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Aug 30 14:54:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:55:02.957615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=18 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:03.023844 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=16 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:05.746767 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba283d2-5600-4417-bf56-86c0896f06fa None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:05.747367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba283d2-5600-4417-bf56-86c0896f06fa None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:05.747481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba283d2-5600-4417-bf56-86c0896f06fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:05.775853 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-0ba283d2-5600-4417-bf56-86c0896f06fa None None] MFA Rules not processed for user `8fc940594a1848aa9644b645049a1bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:05.777303 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0ba283d2-5600-4417-bf56-86c0896f06fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:05.786064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1214/2432] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Aug 30 14:55:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:05.799307 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb8580-3e24-41fd-8d0b-555dd4fe4cb9 service swift] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:05.810929 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb8580-3e24-41fd-8d0b-555dd4fe4cb9 service swift] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:05.817386 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5d446-963a-4773-84a2-c51ce6e82e8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d98f5adfaf410580481417e7ba1638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:05.818127 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5d446-963a-4773-84a2-c51ce6e82e8e service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:05.818401 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5d446-963a-4773-84a2-c51ce6e82e8e service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:05.818664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5d446-963a-4773-84a2-c51ce6e82e8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:05.828154 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5d446-963a-4773-84a2-c51ce6e82e8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:05.829740 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5d446-963a-4773-84a2-c51ce6e82e8e service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:05.838494 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1219/2433] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Aug 30 14:55:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:55:15.623878 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba283d2-5600-4417-bf56-86c0896f06fa None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:15.626313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0ba283d2-5600-4417-bf56-86c0896f06fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:15.657251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba283d2-5600-4417-bf56-86c0896f06fa None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:15.670105 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9622c1-148a-468a-84b1-649cf13eef56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:15.670748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9622c1-148a-468a-84b1-649cf13eef56 service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:15.670986 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9622c1-148a-468a-84b1-649cf13eef56 service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:15.676451 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9622c1-148a-468a-84b1-649cf13eef56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:15.686833 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2a9622c1-148a-468a-84b1-649cf13eef56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:15.715512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9622c1-148a-468a-84b1-649cf13eef56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:15.717208 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9622c1-148a-468a-84b1-649cf13eef56 service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:15.737514 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1215/2434] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:55:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:55:15.834810 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=6 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:20.190679 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374ed55-9d14-401b-ae71-c8181a799165 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:20.191618 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374ed55-9d14-401b-ae71-c8181a799165 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:20.192120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374ed55-9d14-401b-ae71-c8181a799165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:20.213275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f374ed55-9d14-401b-ae71-c8181a799165 None None] MFA Rules not processed for user `06ab23cfa18e47c9871c6f451b871ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:20.214624 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f374ed55-9d14-401b-ae71-c8181a799165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:20.282359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1220/2435] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:20.742060 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9622c1-148a-468a-84b1-649cf13eef56 service nova] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:20.754130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9622c1-148a-468a-84b1-649cf13eef56 service nova] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:20.759324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387033f5-7ad7-445f-b6d9-d793f597dcfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:20.759959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387033f5-7ad7-445f-b6d9-d793f597dcfa service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:20.760263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387033f5-7ad7-445f-b6d9-d793f597dcfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:20.761482 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387033f5-7ad7-445f-b6d9-d793f597dcfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:20.764520 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-387033f5-7ad7-445f-b6d9-d793f597dcfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:20.779603 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387033f5-7ad7-445f-b6d9-d793f597dcfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:20.781304 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387033f5-7ad7-445f-b6d9-d793f597dcfa service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:20.795931 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1216/2436] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:55:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:55:21.030189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47683be7-3f1d-4676-9596-733abf9d92f7 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:21.030640 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47683be7-3f1d-4676-9596-733abf9d92f7 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:21.031013 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47683be7-3f1d-4676-9596-733abf9d92f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:21.056785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-47683be7-3f1d-4676-9596-733abf9d92f7 None None] MFA Rules not processed for user `f72e1a846e864aa8b00edd8ae1a948ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:21.058240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-47683be7-3f1d-4676-9596-733abf9d92f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:21.138877 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1221/2437] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:21.616522 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387033f5-7ad7-445f-b6d9-d793f597dcfa service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:21.634963 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387033f5-7ad7-445f-b6d9-d793f597dcfa service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:21.642683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43286026-9c57-4742-b555-13cf378b1d7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:21.643350 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43286026-9c57-4742-b555-13cf378b1d7d service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:21.643982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43286026-9c57-4742-b555-13cf378b1d7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:21.644274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43286026-9c57-4742-b555-13cf378b1d7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:21.654426 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-43286026-9c57-4742-b555-13cf378b1d7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:21.674571 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43286026-9c57-4742-b555-13cf378b1d7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:21.677510 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43286026-9c57-4742-b555-13cf378b1d7d service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:21.695972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1217/2438] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:55:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:55:23.062246 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4631689-eee4-4c14-9894-9407a779911b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:23.062713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4631689-eee4-4c14-9894-9407a779911b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:23.062822 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4631689-eee4-4c14-9894-9407a779911b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:23.083473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e4631689-eee4-4c14-9894-9407a779911b None None] MFA Rules not processed for user `8fc940594a1848aa9644b645049a1bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:23.084741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e4631689-eee4-4c14-9894-9407a779911b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:23.093025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1222/2439] 10.222.0.25 () {58 vars in 967 bytes} [Tue Aug 30 14:55:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:23.105748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43286026-9c57-4742-b555-13cf378b1d7d service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:23.128477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43286026-9c57-4742-b555-13cf378b1d7d service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:23.134388 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736aa330-1c16-4bf7-bd7f-3ee1320568f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d98f5adfaf410580481417e7ba1638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:23.135000 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736aa330-1c16-4bf7-bd7f-3ee1320568f4 service swift] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:23.135278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736aa330-1c16-4bf7-bd7f-3ee1320568f4 service swift] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:23.135551 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736aa330-1c16-4bf7-bd7f-3ee1320568f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:23.144802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736aa330-1c16-4bf7-bd7f-3ee1320568f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:23.147048 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736aa330-1c16-4bf7-bd7f-3ee1320568f4 service swift] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:23.155768 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1218/2440] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Aug 30 14:55:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:55:25.328025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4631689-eee4-4c14-9894-9407a779911b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:25.343085 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4631689-eee4-4c14-9894-9407a779911b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:25.348473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e9c51-6bf6-4bca-afec-7af72c9a7c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:25.349547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e9c51-6bf6-4bca-afec-7af72c9a7c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:25.349997 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e9c51-6bf6-4bca-afec-7af72c9a7c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:25.350258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e9c51-6bf6-4bca-afec-7af72c9a7c37 admin admin] PATH_INFO: `/v3/users/a20233e0c38e492bad530401f388f19a` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:25.351677 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e9c51-6bf6-4bca-afec-7af72c9a7c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a20233e0c38e492bad530401f388f19a)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:25.355313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e9c51-6bf6-4bca-afec-7af72c9a7c37 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:25.434572 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-374e9c51-6bf6-4bca-afec-7af72c9a7c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a20233e0c38e492bad530401f388f19a'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:25.446974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-374e9c51-6bf6-4bca-afec-7af72c9a7c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a20233e0c38e492bad530401f388f19a'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:25.459512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-374e9c51-6bf6-4bca-afec-7af72c9a7c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a20233e0c38e492bad530401f388f19a'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:25.460550 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-374e9c51-6bf6-4bca-afec-7af72c9a7c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a20233e0c38e492bad530401f388f19a'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:25.480758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-374e9c51-6bf6-4bca-afec-7af72c9a7c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a20233e0c38e492bad530401f388f19a'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:25.488942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1223/2441] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:55:25 2022] DELETE /identity/v3/users/a20233e0c38e492bad530401f388f19a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:55:25.713821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736aa330-1c16-4bf7-bd7f-3ee1320568f4 service swift] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:25.730311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736aa330-1c16-4bf7-bd7f-3ee1320568f4 service swift] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:25.805386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10895a6-e04e-4c1c-a653-6a11ab51510e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:25.806144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10895a6-e04e-4c1c-a653-6a11ab51510e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:25.806416 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10895a6-e04e-4c1c-a653-6a11ab51510e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:25.806680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10895a6-e04e-4c1c-a653-6a11ab51510e admin admin] PATH_INFO: `/v3/projects/ac849c366ee54767bac65b18976c7085` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:25.807881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10895a6-e04e-4c1c-a653-6a11ab51510e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac849c366ee54767bac65b18976c7085)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:25.809899 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10895a6-e04e-4c1c-a653-6a11ab51510e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:25.903328 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b10895a6-e04e-4c1c-a653-6a11ab51510e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac849c366ee54767bac65b18976c7085', 'request_id': 'req-b10895a6-e04e-4c1c-a653-6a11ab51510e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:25.911528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b10895a6-e04e-4c1c-a653-6a11ab51510e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac849c366ee54767bac65b18976c7085', 'request_id': 'req-b10895a6-e04e-4c1c-a653-6a11ab51510e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:25.926276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b10895a6-e04e-4c1c-a653-6a11ab51510e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac849c366ee54767bac65b18976c7085', 'request_id': 'req-b10895a6-e04e-4c1c-a653-6a11ab51510e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:25.954485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b10895a6-e04e-4c1c-a653-6a11ab51510e admin admin] The token cache is being invalidate because project ac849c366ee54767bac65b18976c7085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:55:25.955114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b10895a6-e04e-4c1c-a653-6a11ab51510e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:25.956139 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b10895a6-e04e-4c1c-a653-6a11ab51510e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:25.961642 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1219/2442] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:55:25 2022] DELETE /identity/v3/projects/ac849c366ee54767bac65b18976c7085 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:25.964184 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5623a94-5bbe-422d-abc1-f5686089b426 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:25.964735 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5623a94-5bbe-422d-abc1-f5686089b426 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:25.965033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5623a94-5bbe-422d-abc1-f5686089b426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:25.991391 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d5623a94-5bbe-422d-abc1-f5686089b426 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:25.992774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d5623a94-5bbe-422d-abc1-f5686089b426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:26.008051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:26.090206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1224/2443] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:26.092549 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c843c5-984f-4bd2-b6f5-6abe71c8ed0f None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:26.093751 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c843c5-984f-4bd2-b6f5-6abe71c8ed0f None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:26.093987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c843c5-984f-4bd2-b6f5-6abe71c8ed0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:26.118003 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-82c843c5-984f-4bd2-b6f5-6abe71c8ed0f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:26.119273 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-82c843c5-984f-4bd2-b6f5-6abe71c8ed0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:26.129040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1220/2444] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:26.133606 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5623a94-5bbe-422d-abc1-f5686089b426 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:26.144255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5623a94-5bbe-422d-abc1-f5686089b426 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:26.152384 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ccd7a-6b8e-46be-97ef-4310066536df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:26.153087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ccd7a-6b8e-46be-97ef-4310066536df admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:26.153318 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ccd7a-6b8e-46be-97ef-4310066536df admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:26.153590 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ccd7a-6b8e-46be-97ef-4310066536df admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:26.154285 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ccd7a-6b8e-46be-97ef-4310066536df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:55:26.154741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ccd7a-6b8e-46be-97ef-4310066536df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:26.156961 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ccd7a-6b8e-46be-97ef-4310066536df admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:26.171137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1225/2445] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:55:26.176195 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c843c5-984f-4bd2-b6f5-6abe71c8ed0f None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:26.186452 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c843c5-984f-4bd2-b6f5-6abe71c8ed0f None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:26.195446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e31222-9357-400d-90b2-9aa3d35803b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:26.198241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e31222-9357-400d-90b2-9aa3d35803b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:26.198241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e31222-9357-400d-90b2-9aa3d35803b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:26.199512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e31222-9357-400d-90b2-9aa3d35803b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:26.200933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e31222-9357-400d-90b2-9aa3d35803b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:26.204501 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e31222-9357-400d-90b2-9aa3d35803b0 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:26.248057 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1221/2446] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:55:26 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:55:26.252777 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ccd7a-6b8e-46be-97ef-4310066536df admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:26.272171 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ccd7a-6b8e-46be-97ef-4310066536df admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:26.340838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d94dfa-f0a0-4d9b-963b-c4db507b4fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:26.341891 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d94dfa-f0a0-4d9b-963b-c4db507b4fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:26.342375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d94dfa-f0a0-4d9b-963b-c4db507b4fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:26.342861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d94dfa-f0a0-4d9b-963b-c4db507b4fc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:26.343803 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d94dfa-f0a0-4d9b-963b-c4db507b4fc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:26.348010 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d94dfa-f0a0-4d9b-963b-c4db507b4fc7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:26.376781 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1226/2447] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:55:26 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:55:26.378096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e31222-9357-400d-90b2-9aa3d35803b0 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:26.384161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:26.390736 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e31222-9357-400d-90b2-9aa3d35803b0 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:26.399746 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15094628-0b44-4690-a8b1-ddc7200eff06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:26.400872 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15094628-0b44-4690-a8b1-ddc7200eff06 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:26.401213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15094628-0b44-4690-a8b1-ddc7200eff06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:26.401569 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15094628-0b44-4690-a8b1-ddc7200eff06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:26.402283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15094628-0b44-4690-a8b1-ddc7200eff06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:26.403962 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15094628-0b44-4690-a8b1-ddc7200eff06 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:26.415897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1222/2448] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:55:26.420304 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d94dfa-f0a0-4d9b-963b-c4db507b4fc7 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:26.431605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d94dfa-f0a0-4d9b-963b-c4db507b4fc7 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:26.436621 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb2b8a-e274-4d8b-92f2-bf1d0d7b713f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:26.437279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb2b8a-e274-4d8b-92f2-bf1d0d7b713f admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:26.437527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb2b8a-e274-4d8b-92f2-bf1d0d7b713f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:26.437797 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb2b8a-e274-4d8b-92f2-bf1d0d7b713f admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:26.438466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb2b8a-e274-4d8b-92f2-bf1d0d7b713f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:26.440118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb2b8a-e274-4d8b-92f2-bf1d0d7b713f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:26.457532 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1227/2449] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:55:26.459197 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15094628-0b44-4690-a8b1-ddc7200eff06 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:26.470459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15094628-0b44-4690-a8b1-ddc7200eff06 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:26.476087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba0ab5-2a46-4009-bf67-ac503a096fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:26.477536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba0ab5-2a46-4009-bf67-ac503a096fae admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:26.477811 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba0ab5-2a46-4009-bf67-ac503a096fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:26.478077 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba0ab5-2a46-4009-bf67-ac503a096fae admin admin] PATH_INFO: `/v3/projects/f6ce53558466411292626efed88183b0/users/28b133b471d64de1b9088e6ab8a4c62d/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:26.496368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba0ab5-2a46-4009-bf67-ac503a096fae admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ce53558466411292626efed88183b0, user_id=28b133b471d64de1b9088e6ab8a4c62d, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:26.499329 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba0ab5-2a46-4009-bf67-ac503a096fae admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:26.517626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1223/2450] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:55:26 2022] PUT /identity/v3/projects/f6ce53558466411292626efed88183b0/users/28b133b471d64de1b9088e6ab8a4c62d/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:26.523506 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d772b40-da05-4e9c-a9c9-b623f1706a8c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:26.523918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d772b40-da05-4e9c-a9c9-b623f1706a8c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:26.524225 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d772b40-da05-4e9c-a9c9-b623f1706a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:26.560418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-3d772b40-da05-4e9c-a9c9-b623f1706a8c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:26.561938 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3d772b40-da05-4e9c-a9c9-b623f1706a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:26.654916 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1228/2451] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:26.665664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba0ab5-2a46-4009-bf67-ac503a096fae admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:26.667481 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5dba0ab5-2a46-4009-bf67-ac503a096fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:26.734113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba0ab5-2a46-4009-bf67-ac503a096fae admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:26.738517 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb44d2-e3f1-4c70-8bf4-77a287b65d0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:26.739201 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb44d2-e3f1-4c70-8bf4-77a287b65d0e service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:26.739502 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb44d2-e3f1-4c70-8bf4-77a287b65d0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:26.739769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb44d2-e3f1-4c70-8bf4-77a287b65d0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:26.750499 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb44d2-e3f1-4c70-8bf4-77a287b65d0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:26.751944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb44d2-e3f1-4c70-8bf4-77a287b65d0e service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:26.762821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1224/2452] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:55:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:55:31.943291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c4990-c403-4c66-adc1-81df6d6b8cc7 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:31.943846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c4990-c403-4c66-adc1-81df6d6b8cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:31.943933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c4990-c403-4c66-adc1-81df6d6b8cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:31.983075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4b7c4990-c403-4c66-adc1-81df6d6b8cc7 None None] MFA Rules not processed for user `28b133b471d64de1b9088e6ab8a4c62d`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:31.984814 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4b7c4990-c403-4c66-adc1-81df6d6b8cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:31.998349 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:32.079091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1229/2453] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:32.080766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb44d2-e3f1-4c70-8bf4-77a287b65d0e service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:32.093040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb44d2-e3f1-4c70-8bf4-77a287b65d0e service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:32.100884 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bade3d3-b91c-4066-9fe6-4c6d2b9d809e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:32.101630 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bade3d3-b91c-4066-9fe6-4c6d2b9d809e admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:32.102061 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bade3d3-b91c-4066-9fe6-4c6d2b9d809e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:32.102435 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bade3d3-b91c-4066-9fe6-4c6d2b9d809e admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:32.103852 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bade3d3-b91c-4066-9fe6-4c6d2b9d809e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:32.106532 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bade3d3-b91c-4066-9fe6-4c6d2b9d809e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:32.146565 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1225/2454] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:55:32 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:55:32.150952 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c4990-c403-4c66-adc1-81df6d6b8cc7 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:32.169838 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c4990-c403-4c66-adc1-81df6d6b8cc7 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:32.258800 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e64518-60e6-44ae-996f-0f71d3f5b5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:32.258800 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e64518-60e6-44ae-996f-0f71d3f5b5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:32.258800 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e64518-60e6-44ae-996f-0f71d3f5b5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:32.259268 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e64518-60e6-44ae-996f-0f71d3f5b5d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:32.260297 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e64518-60e6-44ae-996f-0f71d3f5b5d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:32.263247 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e64518-60e6-44ae-996f-0f71d3f5b5d9 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:32.289829 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1230/2455] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:55:32 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:55:32.294025 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bade3d3-b91c-4066-9fe6-4c6d2b9d809e admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:32.309450 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bade3d3-b91c-4066-9fe6-4c6d2b9d809e admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:32.315060 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9b592-977f-40f7-a6c6-bbab03f1ca88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:32.315883 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9b592-977f-40f7-a6c6-bbab03f1ca88 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:32.316310 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9b592-977f-40f7-a6c6-bbab03f1ca88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:32.316645 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9b592-977f-40f7-a6c6-bbab03f1ca88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:32.317416 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9b592-977f-40f7-a6c6-bbab03f1ca88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:32.320068 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9b592-977f-40f7-a6c6-bbab03f1ca88 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:32.337957 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1226/2456] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:55:32.341975 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e64518-60e6-44ae-996f-0f71d3f5b5d9 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:32.354340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e64518-60e6-44ae-996f-0f71d3f5b5d9 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:32.361543 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e59c6-1be0-4b75-bc99-87d663ccb75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:32.362312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e59c6-1be0-4b75-bc99-87d663ccb75d admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:32.362671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e59c6-1be0-4b75-bc99-87d663ccb75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:32.363005 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e59c6-1be0-4b75-bc99-87d663ccb75d admin admin] PATH_INFO: `/v3/projects/9f56558480f446ffb7c52524676231f6/users/119b7bf7dbae42c48426c01bcfb2688a/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:32.388345 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e59c6-1be0-4b75-bc99-87d663ccb75d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f56558480f446ffb7c52524676231f6, user_id=119b7bf7dbae42c48426c01bcfb2688a, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:32.394753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e59c6-1be0-4b75-bc99-87d663ccb75d admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:32.422924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1231/2457] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:55:32 2022] PUT /identity/v3/projects/9f56558480f446ffb7c52524676231f6/users/119b7bf7dbae42c48426c01bcfb2688a/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:32.427592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9b592-977f-40f7-a6c6-bbab03f1ca88 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:32.441079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9b592-977f-40f7-a6c6-bbab03f1ca88 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:32.599815 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120423a6-315d-456d-9888-1c383bcc56a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:32.600358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120423a6-315d-456d-9888-1c383bcc56a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:32.608242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120423a6-315d-456d-9888-1c383bcc56a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:32.608242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120423a6-315d-456d-9888-1c383bcc56a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:32.608242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120423a6-315d-456d-9888-1c383bcc56a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:32.608242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120423a6-315d-456d-9888-1c383bcc56a5 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:32.646139 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1227/2458] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:55:32.648280 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e59c6-1be0-4b75-bc99-87d663ccb75d admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:32.670801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e59c6-1be0-4b75-bc99-87d663ccb75d admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:32.678739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9428b-295c-4cb1-9e75-c0e0bf59d8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:32.681068 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9428b-295c-4cb1-9e75-c0e0bf59d8cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:32.681556 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9428b-295c-4cb1-9e75-c0e0bf59d8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:32.681830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9428b-295c-4cb1-9e75-c0e0bf59d8cd admin admin] PATH_INFO: `/v3/projects/9f56558480f446ffb7c52524676231f6/users/119b7bf7dbae42c48426c01bcfb2688a/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:32.685001 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9428b-295c-4cb1-9e75-c0e0bf59d8cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f56558480f446ffb7c52524676231f6, user_id=119b7bf7dbae42c48426c01bcfb2688a, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:32.687796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9428b-295c-4cb1-9e75-c0e0bf59d8cd admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:32.727690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1232/2459] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:55:32 2022] PUT /identity/v3/projects/9f56558480f446ffb7c52524676231f6/users/119b7bf7dbae42c48426c01bcfb2688a/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:37.701223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120423a6-315d-456d-9888-1c383bcc56a5 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:37.704020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-120423a6-315d-456d-9888-1c383bcc56a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:37.765772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:37.822181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120423a6-315d-456d-9888-1c383bcc56a5 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:37.913332 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8d986-2576-4e56-a0c5-e3ac860db459 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:37.914054 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8d986-2576-4e56-a0c5-e3ac860db459 service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:37.914333 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8d986-2576-4e56-a0c5-e3ac860db459 service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:37.914601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8d986-2576-4e56-a0c5-e3ac860db459 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:37.916477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-66f8d986-2576-4e56-a0c5-e3ac860db459 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:37.932572 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8d986-2576-4e56-a0c5-e3ac860db459 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:37.934296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8d986-2576-4e56-a0c5-e3ac860db459 service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:37.947922 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1228/2460] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:55:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:55:39.224173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1330e-952c-4a2b-927d-4eb610a207ed None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:39.224667 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1330e-952c-4a2b-927d-4eb610a207ed None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:39.224754 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1330e-952c-4a2b-927d-4eb610a207ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:39.255210 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1ee1330e-952c-4a2b-927d-4eb610a207ed None None] MFA Rules not processed for user `119b7bf7dbae42c48426c01bcfb2688a`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:39.256496 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1ee1330e-952c-4a2b-927d-4eb610a207ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:39.312502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:39.339270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1233/2461] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:39.366168 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8d986-2576-4e56-a0c5-e3ac860db459 service nova] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:39.368203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-66f8d986-2576-4e56-a0c5-e3ac860db459 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:39.387030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8d986-2576-4e56-a0c5-e3ac860db459 service nova] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:39.409912 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b02c9-ecbd-4218-b300-ebebe0cb1254 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:39.410444 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b02c9-ecbd-4218-b300-ebebe0cb1254 service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:39.410965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b02c9-ecbd-4218-b300-ebebe0cb1254 service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:39.411333 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b02c9-ecbd-4218-b300-ebebe0cb1254 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:39.525209 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b02c9-ecbd-4218-b300-ebebe0cb1254 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:39.531323 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b02c9-ecbd-4218-b300-ebebe0cb1254 service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:39.542999 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1229/2462] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:55:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:55:40.761761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1330e-952c-4a2b-927d-4eb610a207ed None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:40.857919 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1330e-952c-4a2b-927d-4eb610a207ed None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:40.862864 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8b3d5-7a55-42f5-ba1f-a0ed0ba0b227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:40.864321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8b3d5-7a55-42f5-ba1f-a0ed0ba0b227 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:40.865054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8b3d5-7a55-42f5-ba1f-a0ed0ba0b227 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:40.865515 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8b3d5-7a55-42f5-ba1f-a0ed0ba0b227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:40.868156 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-04b8b3d5-7a55-42f5-ba1f-a0ed0ba0b227 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:40.993475 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8b3d5-7a55-42f5-ba1f-a0ed0ba0b227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:40.994977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8b3d5-7a55-42f5-ba1f-a0ed0ba0b227 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:41.007158 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1234/2463] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:55:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:55:41.109187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8327c-b600-420f-a435-12bef53387b1 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:41.110394 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8327c-b600-420f-a435-12bef53387b1 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:41.110394 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8327c-b600-420f-a435-12bef53387b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:41.135515 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-d3d8327c-b600-420f-a435-12bef53387b1 None None] MFA Rules not processed for user `28b133b471d64de1b9088e6ab8a4c62d`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:41.137933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d3d8327c-b600-420f-a435-12bef53387b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:41.148899 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1230/2464] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:45.503518 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8b3d5-7a55-42f5-ba1f-a0ed0ba0b227 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:45.505128 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-04b8b3d5-7a55-42f5-ba1f-a0ed0ba0b227 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:45.529812 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8b3d5-7a55-42f5-ba1f-a0ed0ba0b227 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:45.534185 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f36db-300a-44c7-bc11-c7699acb6d1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:45.535052 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f36db-300a-44c7-bc11-c7699acb6d1d service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:45.535604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f36db-300a-44c7-bc11-c7699acb6d1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:45.536129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f36db-300a-44c7-bc11-c7699acb6d1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:45.538206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-733f36db-300a-44c7-bc11-c7699acb6d1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:45.640509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f36db-300a-44c7-bc11-c7699acb6d1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:45.642277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f36db-300a-44c7-bc11-c7699acb6d1d service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:45.658104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1235/2465] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:55:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:55:45.897502 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8327c-b600-420f-a435-12bef53387b1 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:45.899014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d3d8327c-b600-420f-a435-12bef53387b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:45.913655 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8327c-b600-420f-a435-12bef53387b1 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:45.932554 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb40d1c-7f36-47a9-892e-df241d8d9159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:45.933216 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb40d1c-7f36-47a9-892e-df241d8d9159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:45.933448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb40d1c-7f36-47a9-892e-df241d8d9159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:45.933804 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb40d1c-7f36-47a9-892e-df241d8d9159 admin admin] PATH_INFO: `/v3/users/a1b3f34373414c61b94e2a8d9018cbd5` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:45.958145 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb40d1c-7f36-47a9-892e-df241d8d9159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1b3f34373414c61b94e2a8d9018cbd5)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:45.960348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb40d1c-7f36-47a9-892e-df241d8d9159 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:46.014698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:46.031313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bfb40d1c-7f36-47a9-892e-df241d8d9159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1b3f34373414c61b94e2a8d9018cbd5'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:46.032462 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bfb40d1c-7f36-47a9-892e-df241d8d9159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1b3f34373414c61b94e2a8d9018cbd5'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:46.033309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bfb40d1c-7f36-47a9-892e-df241d8d9159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1b3f34373414c61b94e2a8d9018cbd5'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:46.049465 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bfb40d1c-7f36-47a9-892e-df241d8d9159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1b3f34373414c61b94e2a8d9018cbd5'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:46.072952 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bfb40d1c-7f36-47a9-892e-df241d8d9159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1b3f34373414c61b94e2a8d9018cbd5'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:46.091542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1231/2466] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:55:45 2022] DELETE /identity/v3/users/a1b3f34373414c61b94e2a8d9018cbd5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:46.342635 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f36db-300a-44c7-bc11-c7699acb6d1d service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:46.369028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f36db-300a-44c7-bc11-c7699acb6d1d service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:46.431041 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:46.469980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d397e8-2c9d-4256-ae48-cea0cd50dd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:46.470665 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d397e8-2c9d-4256-ae48-cea0cd50dd2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:46.471067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d397e8-2c9d-4256-ae48-cea0cd50dd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:46.471242 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d397e8-2c9d-4256-ae48-cea0cd50dd2a admin admin] PATH_INFO: `/v3/projects/aa5fb93320db4b1a838bed8b14ca5653` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:46.486776 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d397e8-2c9d-4256-ae48-cea0cd50dd2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa5fb93320db4b1a838bed8b14ca5653)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:46.488878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d397e8-2c9d-4256-ae48-cea0cd50dd2a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:46.585143 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-73d397e8-2c9d-4256-ae48-cea0cd50dd2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa5fb93320db4b1a838bed8b14ca5653', 'request_id': 'req-73d397e8-2c9d-4256-ae48-cea0cd50dd2a'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:46.602105 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-73d397e8-2c9d-4256-ae48-cea0cd50dd2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa5fb93320db4b1a838bed8b14ca5653', 'request_id': 'req-73d397e8-2c9d-4256-ae48-cea0cd50dd2a'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:46.612312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-73d397e8-2c9d-4256-ae48-cea0cd50dd2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa5fb93320db4b1a838bed8b14ca5653', 'request_id': 'req-73d397e8-2c9d-4256-ae48-cea0cd50dd2a'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:46.628791 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-73d397e8-2c9d-4256-ae48-cea0cd50dd2a admin admin] The token cache is being invalidate because project aa5fb93320db4b1a838bed8b14ca5653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:55:46.629604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-73d397e8-2c9d-4256-ae48-cea0cd50dd2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:46.631226 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-73d397e8-2c9d-4256-ae48-cea0cd50dd2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:46.633617 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1236/2467] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:55:46 2022] DELETE /identity/v3/projects/aa5fb93320db4b1a838bed8b14ca5653 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:46.639152 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1010c-d490-48cf-840a-1d554e8140d7 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:46.639669 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1010c-d490-48cf-840a-1d554e8140d7 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:46.640049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1010c-d490-48cf-840a-1d554e8140d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:46.667908 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-cef1010c-d490-48cf-840a-1d554e8140d7 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:46.669483 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-cef1010c-d490-48cf-840a-1d554e8140d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:46.755088 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1232/2468] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:46.764797 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f8648-06e6-47d1-95e9-d42914ffb7c0 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:46.765438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f8648-06e6-47d1-95e9-d42914ffb7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:46.765749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f8648-06e6-47d1-95e9-d42914ffb7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:46.796921 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8a2f8648-06e6-47d1-95e9-d42914ffb7c0 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:46.798435 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8a2f8648-06e6-47d1-95e9-d42914ffb7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:46.809137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1237/2469] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:46.814857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1010c-d490-48cf-840a-1d554e8140d7 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:46.828681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1010c-d490-48cf-840a-1d554e8140d7 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:46.834758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945de20-867b-4f87-912c-981291d71ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:46.835486 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945de20-867b-4f87-912c-981291d71ddd admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:46.835768 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945de20-867b-4f87-912c-981291d71ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:46.836068 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945de20-867b-4f87-912c-981291d71ddd admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:46.836756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945de20-867b-4f87-912c-981291d71ddd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:55:46.837108 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945de20-867b-4f87-912c-981291d71ddd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:46.838917 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945de20-867b-4f87-912c-981291d71ddd admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:46.865502 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1233/2470] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:55:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:55:46.869656 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f8648-06e6-47d1-95e9-d42914ffb7c0 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:46.883512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f8648-06e6-47d1-95e9-d42914ffb7c0 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:46.896887 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885d7df-2163-4a10-aa8b-51bb5b8a330f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:46.897534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885d7df-2163-4a10-aa8b-51bb5b8a330f admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:46.897824 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885d7df-2163-4a10-aa8b-51bb5b8a330f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:46.898088 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885d7df-2163-4a10-aa8b-51bb5b8a330f admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:46.898781 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885d7df-2163-4a10-aa8b-51bb5b8a330f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:46.901086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885d7df-2163-4a10-aa8b-51bb5b8a330f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:46.947991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1238/2471] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:55:46 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:55:46.948404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945de20-867b-4f87-912c-981291d71ddd admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:46.960707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945de20-867b-4f87-912c-981291d71ddd admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:47.039112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f24d6-36ff-44be-bb97-f01378dfe78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:47.039764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f24d6-36ff-44be-bb97-f01378dfe78f admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:47.040074 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f24d6-36ff-44be-bb97-f01378dfe78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:47.040347 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f24d6-36ff-44be-bb97-f01378dfe78f admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:47.040988 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f24d6-36ff-44be-bb97-f01378dfe78f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:47.042905 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f24d6-36ff-44be-bb97-f01378dfe78f admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:47.068300 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1234/2472] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:55:46 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:55:47.073566 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885d7df-2163-4a10-aa8b-51bb5b8a330f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:47.085603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885d7df-2163-4a10-aa8b-51bb5b8a330f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:47.090837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac95adc-d3a0-4c85-8940-5d59feb31759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:47.091861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac95adc-d3a0-4c85-8940-5d59feb31759 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:47.092122 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac95adc-d3a0-4c85-8940-5d59feb31759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:47.092497 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac95adc-d3a0-4c85-8940-5d59feb31759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:47.093375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac95adc-d3a0-4c85-8940-5d59feb31759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:47.095745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac95adc-d3a0-4c85-8940-5d59feb31759 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:47.107799 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1239/2473] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:55:47.112794 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f24d6-36ff-44be-bb97-f01378dfe78f admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:47.125120 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f24d6-36ff-44be-bb97-f01378dfe78f admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:47.132112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee9ea7-fde1-4e97-a522-421cd464b0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:47.133540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee9ea7-fde1-4e97-a522-421cd464b0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:47.133891 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee9ea7-fde1-4e97-a522-421cd464b0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:47.134221 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee9ea7-fde1-4e97-a522-421cd464b0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:47.135229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee9ea7-fde1-4e97-a522-421cd464b0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:47.136920 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee9ea7-fde1-4e97-a522-421cd464b0b4 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:47.149849 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1235/2474] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:55:47.153872 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac95adc-d3a0-4c85-8940-5d59feb31759 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:47.167988 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac95adc-d3a0-4c85-8940-5d59feb31759 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:47.182014 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e3b87-cd42-4f41-a7e5-13a3ea1e1c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:47.182865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e3b87-cd42-4f41-a7e5-13a3ea1e1c76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:47.183260 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e3b87-cd42-4f41-a7e5-13a3ea1e1c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:47.183616 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e3b87-cd42-4f41-a7e5-13a3ea1e1c76 admin admin] PATH_INFO: `/v3/projects/b094bc25326140a68545bb8449b53fec/users/7e8ab1af104c4a67b02f000fa8912063/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:47.200943 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e3b87-cd42-4f41-a7e5-13a3ea1e1c76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b094bc25326140a68545bb8449b53fec, user_id=7e8ab1af104c4a67b02f000fa8912063, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:47.203538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e3b87-cd42-4f41-a7e5-13a3ea1e1c76 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:47.220433 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1240/2475] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:55:47 2022] PUT /identity/v3/projects/b094bc25326140a68545bb8449b53fec/users/7e8ab1af104c4a67b02f000fa8912063/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:47.231327 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad946d9-b643-45d1-abe2-e33cb657f684 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:47.231607 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad946d9-b643-45d1-abe2-e33cb657f684 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:47.231881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad946d9-b643-45d1-abe2-e33cb657f684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:47.263754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-9ad946d9-b643-45d1-abe2-e33cb657f684 None None] MFA Rules not processed for user `7e8ab1af104c4a67b02f000fa8912063`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:47.265029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-9ad946d9-b643-45d1-abe2-e33cb657f684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:47.270887 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:47.350717 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1236/2476] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:47.366304 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e3b87-cd42-4f41-a7e5-13a3ea1e1c76 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:47.368341 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e81e3b87-cd42-4f41-a7e5-13a3ea1e1c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:47.428103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:47.457355 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e3b87-cd42-4f41-a7e5-13a3ea1e1c76 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:47.464177 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16d553-691d-4e2f-8e28-34f1937fabc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:47.464958 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16d553-691d-4e2f-8e28-34f1937fabc0 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:47.465240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16d553-691d-4e2f-8e28-34f1937fabc0 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:47.465510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16d553-691d-4e2f-8e28-34f1937fabc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:47.474426 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16d553-691d-4e2f-8e28-34f1937fabc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:47.476227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16d553-691d-4e2f-8e28-34f1937fabc0 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:47.487561 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1241/2477] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:55:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:55:50.318878 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad946d9-b643-45d1-abe2-e33cb657f684 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:50.416252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad946d9-b643-45d1-abe2-e33cb657f684 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:50.420852 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb1119-6ba0-490d-b7f3-8c3f967a85fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:50.421498 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb1119-6ba0-490d-b7f3-8c3f967a85fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:50.421778 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb1119-6ba0-490d-b7f3-8c3f967a85fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:50.422043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb1119-6ba0-490d-b7f3-8c3f967a85fb admin admin] PATH_INFO: `/v3/users/7e8ab1af104c4a67b02f000fa8912063` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:50.423191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb1119-6ba0-490d-b7f3-8c3f967a85fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e8ab1af104c4a67b02f000fa8912063)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:50.425207 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb1119-6ba0-490d-b7f3-8c3f967a85fb admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:50.487924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cfeb1119-6ba0-490d-b7f3-8c3f967a85fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e8ab1af104c4a67b02f000fa8912063'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:50.488777 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cfeb1119-6ba0-490d-b7f3-8c3f967a85fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e8ab1af104c4a67b02f000fa8912063'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:50.489421 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cfeb1119-6ba0-490d-b7f3-8c3f967a85fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e8ab1af104c4a67b02f000fa8912063'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:50.499459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cfeb1119-6ba0-490d-b7f3-8c3f967a85fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e8ab1af104c4a67b02f000fa8912063'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:50.512313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cfeb1119-6ba0-490d-b7f3-8c3f967a85fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e8ab1af104c4a67b02f000fa8912063'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:50.529438 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1237/2478] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:55:50 2022] DELETE /identity/v3/users/7e8ab1af104c4a67b02f000fa8912063 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:50.534135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfef54d-b9a3-4387-8616-784620fd7286 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:50.534677 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfef54d-b9a3-4387-8616-784620fd7286 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:50.534782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfef54d-b9a3-4387-8616-784620fd7286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:50.560641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-cdfef54d-b9a3-4387-8616-784620fd7286 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:50.561906 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cdfef54d-b9a3-4387-8616-784620fd7286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:50.627051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1242/2479] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:50.639159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb1119-6ba0-490d-b7f3-8c3f967a85fb admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:50.640809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-cfeb1119-6ba0-490d-b7f3-8c3f967a85fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:50.685508 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:50.722858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb1119-6ba0-490d-b7f3-8c3f967a85fb admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:50.727403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730339fb-162c-4e76-9f14-338c153eea89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:50.728089 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730339fb-162c-4e76-9f14-338c153eea89 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:50.728359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730339fb-162c-4e76-9f14-338c153eea89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:50.728629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730339fb-162c-4e76-9f14-338c153eea89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:50.738841 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730339fb-162c-4e76-9f14-338c153eea89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:50.740369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730339fb-162c-4e76-9f14-338c153eea89 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:50.752375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1238/2480] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:55:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:55:51.060130 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfef54d-b9a3-4387-8616-784620fd7286 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:51.061932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cdfef54d-b9a3-4387-8616-784620fd7286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:51.077783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfef54d-b9a3-4387-8616-784620fd7286 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:51.084626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfd50d-f869-4478-8702-2b98866324b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:51.084626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfd50d-f869-4478-8702-2b98866324b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:51.084626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfd50d-f869-4478-8702-2b98866324b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:51.085090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfd50d-f869-4478-8702-2b98866324b5 admin admin] PATH_INFO: `/v3/projects/b094bc25326140a68545bb8449b53fec` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:51.086670 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfd50d-f869-4478-8702-2b98866324b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b094bc25326140a68545bb8449b53fec)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:51.088821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfd50d-f869-4478-8702-2b98866324b5 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:51.147252 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9cdfd50d-f869-4478-8702-2b98866324b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b094bc25326140a68545bb8449b53fec', 'request_id': 'req-9cdfd50d-f869-4478-8702-2b98866324b5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:51.157693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9cdfd50d-f869-4478-8702-2b98866324b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b094bc25326140a68545bb8449b53fec', 'request_id': 'req-9cdfd50d-f869-4478-8702-2b98866324b5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:51.164345 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9cdfd50d-f869-4478-8702-2b98866324b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b094bc25326140a68545bb8449b53fec', 'request_id': 'req-9cdfd50d-f869-4478-8702-2b98866324b5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:51.179824 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9cdfd50d-f869-4478-8702-2b98866324b5 admin admin] The token cache is being invalidate because project b094bc25326140a68545bb8449b53fec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:55:51.180121 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9cdfd50d-f869-4478-8702-2b98866324b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:51.181022 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9cdfd50d-f869-4478-8702-2b98866324b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:51.183020 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1243/2481] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:55:51 2022] DELETE /identity/v3/projects/b094bc25326140a68545bb8449b53fec => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:51.188304 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138083a-6643-4107-9c91-aed2f5627417 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:51.188592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138083a-6643-4107-9c91-aed2f5627417 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:51.188868 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138083a-6643-4107-9c91-aed2f5627417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:51.210243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-4138083a-6643-4107-9c91-aed2f5627417 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:51.211506 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4138083a-6643-4107-9c91-aed2f5627417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:51.283323 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1239/2482] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:51.292095 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd7b01-34af-41dd-80ef-9d14073ce171 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:51.292359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd7b01-34af-41dd-80ef-9d14073ce171 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:51.292625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd7b01-34af-41dd-80ef-9d14073ce171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:51.315357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e1fd7b01-34af-41dd-80ef-9d14073ce171 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:51.316672 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e1fd7b01-34af-41dd-80ef-9d14073ce171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:51.327817 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1244/2483] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:51.332531 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138083a-6643-4107-9c91-aed2f5627417 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:51.343461 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138083a-6643-4107-9c91-aed2f5627417 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:51.349400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf5bf2-ea41-4b39-9b4e-98ff2885c829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:51.350192 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf5bf2-ea41-4b39-9b4e-98ff2885c829 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:51.350469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf5bf2-ea41-4b39-9b4e-98ff2885c829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:51.350733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf5bf2-ea41-4b39-9b4e-98ff2885c829 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:51.351401 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf5bf2-ea41-4b39-9b4e-98ff2885c829 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:55:51.351700 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf5bf2-ea41-4b39-9b4e-98ff2885c829 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:51.353360 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf5bf2-ea41-4b39-9b4e-98ff2885c829 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:51.369751 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1240/2484] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:55:51.374242 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd7b01-34af-41dd-80ef-9d14073ce171 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:51.386120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd7b01-34af-41dd-80ef-9d14073ce171 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:51.390333 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4f6ff-5bc3-4cfc-8b7a-f9233c757828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:51.391141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4f6ff-5bc3-4cfc-8b7a-f9233c757828 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:51.391414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4f6ff-5bc3-4cfc-8b7a-f9233c757828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:51.391674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4f6ff-5bc3-4cfc-8b7a-f9233c757828 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:51.392353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4f6ff-5bc3-4cfc-8b7a-f9233c757828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:51.394304 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4f6ff-5bc3-4cfc-8b7a-f9233c757828 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:51.425182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1245/2485] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:55:51 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:55:51.432976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf5bf2-ea41-4b39-9b4e-98ff2885c829 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:51.446951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf5bf2-ea41-4b39-9b4e-98ff2885c829 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:51.509090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2f3c1-aaf0-4e88-9d6e-cd1c05fdc766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:51.509880 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2f3c1-aaf0-4e88-9d6e-cd1c05fdc766 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:51.510204 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2f3c1-aaf0-4e88-9d6e-cd1c05fdc766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:51.510620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2f3c1-aaf0-4e88-9d6e-cd1c05fdc766 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:51.511474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2f3c1-aaf0-4e88-9d6e-cd1c05fdc766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:51.513874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2f3c1-aaf0-4e88-9d6e-cd1c05fdc766 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:51.542330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1241/2486] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:55:51 2022] POST /identity/v3/users => generated 303 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:55:51.546943 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4f6ff-5bc3-4cfc-8b7a-f9233c757828 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:51.559226 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4f6ff-5bc3-4cfc-8b7a-f9233c757828 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:51.564681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0104f122-9c5e-423e-b9e7-d86348ea9fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:51.565306 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104f122-9c5e-423e-b9e7-d86348ea9fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:51.565582 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104f122-9c5e-423e-b9e7-d86348ea9fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:51.565846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104f122-9c5e-423e-b9e7-d86348ea9fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:51.566500 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104f122-9c5e-423e-b9e7-d86348ea9fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:51.568393 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104f122-9c5e-423e-b9e7-d86348ea9fb2 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:51.582173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1246/2487] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:55:51.587049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2f3c1-aaf0-4e88-9d6e-cd1c05fdc766 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:51.598559 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2f3c1-aaf0-4e88-9d6e-cd1c05fdc766 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:51.604448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc9e39-f4d2-4995-8683-256582aae627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:51.605078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc9e39-f4d2-4995-8683-256582aae627 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:51.605354 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc9e39-f4d2-4995-8683-256582aae627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:51.605585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc9e39-f4d2-4995-8683-256582aae627 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:51.606424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc9e39-f4d2-4995-8683-256582aae627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:51.608263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc9e39-f4d2-4995-8683-256582aae627 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:51.621722 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1242/2488] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:55:51.626701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0104f122-9c5e-423e-b9e7-d86348ea9fb2 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:51.638949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0104f122-9c5e-423e-b9e7-d86348ea9fb2 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:51.643922 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662910fc-1575-4dd8-9eca-8b7279e9dcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:51.644620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662910fc-1575-4dd8-9eca-8b7279e9dcc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:51.644939 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662910fc-1575-4dd8-9eca-8b7279e9dcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:51.645210 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662910fc-1575-4dd8-9eca-8b7279e9dcc8 admin admin] PATH_INFO: `/v3/projects/4eaa602c80cd44fbb764ce5a52c05a57/users/d543e04a4da34a159dbe3d66b9ddd012/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:51.662429 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662910fc-1575-4dd8-9eca-8b7279e9dcc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eaa602c80cd44fbb764ce5a52c05a57, user_id=d543e04a4da34a159dbe3d66b9ddd012, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:51.665080 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662910fc-1575-4dd8-9eca-8b7279e9dcc8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:51.668278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:51.684448 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1247/2489] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:55:51 2022] PUT /identity/v3/projects/4eaa602c80cd44fbb764ce5a52c05a57/users/d543e04a4da34a159dbe3d66b9ddd012/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:51.689928 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ed23a-4ad7-476f-b3ea-8328bdcf4cb5 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:51.690340 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ed23a-4ad7-476f-b3ea-8328bdcf4cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:51.690763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ed23a-4ad7-476f-b3ea-8328bdcf4cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:51.710975 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-162ed23a-4ad7-476f-b3ea-8328bdcf4cb5 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:51.712251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-162ed23a-4ad7-476f-b3ea-8328bdcf4cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:51.744085 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:51.787550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1243/2490] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:51.799263 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662910fc-1575-4dd8-9eca-8b7279e9dcc8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:51.801602 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-662910fc-1575-4dd8-9eca-8b7279e9dcc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:51.867984 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662910fc-1575-4dd8-9eca-8b7279e9dcc8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:51.871781 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67836a-82de-4774-9a03-aeb26285f93e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:51.872435 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67836a-82de-4774-9a03-aeb26285f93e service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:51.872710 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67836a-82de-4774-9a03-aeb26285f93e service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:51.872981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67836a-82de-4774-9a03-aeb26285f93e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:51.885301 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67836a-82de-4774-9a03-aeb26285f93e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:51.887075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67836a-82de-4774-9a03-aeb26285f93e service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:51.898841 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1248/2491] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:55:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:55:52.880753 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081259b2-e3b8-43c2-9abb-7f4b061a74ea None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:52.881352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081259b2-e3b8-43c2-9abb-7f4b061a74ea None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:52.881734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081259b2-e3b8-43c2-9abb-7f4b061a74ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:52.919282 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-081259b2-e3b8-43c2-9abb-7f4b061a74ea None None] MFA Rules not processed for user `d543e04a4da34a159dbe3d66b9ddd012`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:52.920562 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-081259b2-e3b8-43c2-9abb-7f4b061a74ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:52.995072 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1244/2492] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:52 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:52.999709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67836a-82de-4774-9a03-aeb26285f93e service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:53.009973 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67836a-82de-4774-9a03-aeb26285f93e service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:53.015016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c77017-8af7-42f1-b907-94f6c2aab64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:53.015732 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c77017-8af7-42f1-b907-94f6c2aab64c admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:53.016117 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c77017-8af7-42f1-b907-94f6c2aab64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:53.016565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c77017-8af7-42f1-b907-94f6c2aab64c admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:53.017242 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c77017-8af7-42f1-b907-94f6c2aab64c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:53.019224 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c77017-8af7-42f1-b907-94f6c2aab64c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:53.053035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1249/2493] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:55:52 2022] POST /identity/v3/projects => generated 403 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:55:53.057356 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081259b2-e3b8-43c2-9abb-7f4b061a74ea None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:53.068349 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081259b2-e3b8-43c2-9abb-7f4b061a74ea None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:53.134202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810da470-7791-4e4f-be71-99e264446034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:53.134816 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810da470-7791-4e4f-be71-99e264446034 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:53.135090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810da470-7791-4e4f-be71-99e264446034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:53.135354 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810da470-7791-4e4f-be71-99e264446034 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:53.136041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810da470-7791-4e4f-be71-99e264446034 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:53.137943 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810da470-7791-4e4f-be71-99e264446034 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:53.163136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1245/2494] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:55:53 2022] POST /identity/v3/users => generated 308 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:55:53.167832 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c77017-8af7-42f1-b907-94f6c2aab64c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:53.179284 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c77017-8af7-42f1-b907-94f6c2aab64c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:53.189220 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb12e64-6fab-4d75-ad09-30c7be4ea5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:53.189868 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb12e64-6fab-4d75-ad09-30c7be4ea5c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:53.190148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb12e64-6fab-4d75-ad09-30c7be4ea5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:53.190403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb12e64-6fab-4d75-ad09-30c7be4ea5c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:53.191063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb12e64-6fab-4d75-ad09-30c7be4ea5c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:53.197226 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb12e64-6fab-4d75-ad09-30c7be4ea5c5 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:53.211372 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1250/2495] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:55:53.216078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810da470-7791-4e4f-be71-99e264446034 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:53.227486 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810da470-7791-4e4f-be71-99e264446034 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:53.232323 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7158cf4-b2b7-4aa8-9772-0c3780eb55c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:53.233240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7158cf4-b2b7-4aa8-9772-0c3780eb55c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:53.233628 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7158cf4-b2b7-4aa8-9772-0c3780eb55c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:53.234013 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7158cf4-b2b7-4aa8-9772-0c3780eb55c2 admin admin] PATH_INFO: `/v3/projects/8e84888f58d74331b54c936cd8977b31/users/fc04571f25874214a99973b96c1f4a5d/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:53.252767 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7158cf4-b2b7-4aa8-9772-0c3780eb55c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e84888f58d74331b54c936cd8977b31, user_id=fc04571f25874214a99973b96c1f4a5d, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:53.255489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7158cf4-b2b7-4aa8-9772-0c3780eb55c2 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:53.271328 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1246/2496] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:55:53 2022] PUT /identity/v3/projects/8e84888f58d74331b54c936cd8977b31/users/fc04571f25874214a99973b96c1f4a5d/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:53.277312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb12e64-6fab-4d75-ad09-30c7be4ea5c5 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:53.288134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb12e64-6fab-4d75-ad09-30c7be4ea5c5 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:53.360584 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43287ae3-0bf5-4397-91f2-6bc9c1dab102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:53.361216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43287ae3-0bf5-4397-91f2-6bc9c1dab102 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:53.361495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43287ae3-0bf5-4397-91f2-6bc9c1dab102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:53.361803 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43287ae3-0bf5-4397-91f2-6bc9c1dab102 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:53.362440 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43287ae3-0bf5-4397-91f2-6bc9c1dab102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:53.364070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43287ae3-0bf5-4397-91f2-6bc9c1dab102 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:53.376386 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1251/2497] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:55:53.381528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7158cf4-b2b7-4aa8-9772-0c3780eb55c2 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:53.392801 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7158cf4-b2b7-4aa8-9772-0c3780eb55c2 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:53.398644 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80e1ff-a6e6-47e8-8d7c-928c469f0619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:53.399306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80e1ff-a6e6-47e8-8d7c-928c469f0619 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:53.399583 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80e1ff-a6e6-47e8-8d7c-928c469f0619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:53.399844 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80e1ff-a6e6-47e8-8d7c-928c469f0619 admin admin] PATH_INFO: `/v3/projects/8e84888f58d74331b54c936cd8977b31/users/fc04571f25874214a99973b96c1f4a5d/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:53.402179 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80e1ff-a6e6-47e8-8d7c-928c469f0619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e84888f58d74331b54c936cd8977b31, user_id=fc04571f25874214a99973b96c1f4a5d, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:53.404724 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80e1ff-a6e6-47e8-8d7c-928c469f0619 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:53.420565 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1247/2498] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:55:53 2022] PUT /identity/v3/projects/8e84888f58d74331b54c936cd8977b31/users/fc04571f25874214a99973b96c1f4a5d/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:54.107575 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6b9ce-17c1-441c-8bc4-486ef1b1c290 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:54.108239 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6b9ce-17c1-441c-8bc4-486ef1b1c290 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:54.108239 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6b9ce-17c1-441c-8bc4-486ef1b1c290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:54.148528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-03a6b9ce-17c1-441c-8bc4-486ef1b1c290 None None] MFA Rules not processed for user `fc04571f25874214a99973b96c1f4a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:54.149998 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-03a6b9ce-17c1-441c-8bc4-486ef1b1c290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:54.208383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:54.232188 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1252/2499] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:54 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:55:55.355443 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80e1ff-a6e6-47e8-8d7c-928c469f0619 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:55.377755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:55.437674 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80e1ff-a6e6-47e8-8d7c-928c469f0619 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:55.443547 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dd005-7011-4ba1-b259-374481eff2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:55.444219 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dd005-7011-4ba1-b259-374481eff2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:55.444511 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dd005-7011-4ba1-b259-374481eff2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:55.444751 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dd005-7011-4ba1-b259-374481eff2b8 admin admin] PATH_INFO: `/v3/users/d543e04a4da34a159dbe3d66b9ddd012` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:55.446106 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dd005-7011-4ba1-b259-374481eff2b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d543e04a4da34a159dbe3d66b9ddd012)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:55.448133 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dd005-7011-4ba1-b259-374481eff2b8 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:55.500545 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eb7dd005-7011-4ba1-b259-374481eff2b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd543e04a4da34a159dbe3d66b9ddd012'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:55.501213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eb7dd005-7011-4ba1-b259-374481eff2b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd543e04a4da34a159dbe3d66b9ddd012'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:55.501839 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eb7dd005-7011-4ba1-b259-374481eff2b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd543e04a4da34a159dbe3d66b9ddd012'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:55.512030 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eb7dd005-7011-4ba1-b259-374481eff2b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd543e04a4da34a159dbe3d66b9ddd012'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:55.526897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eb7dd005-7011-4ba1-b259-374481eff2b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd543e04a4da34a159dbe3d66b9ddd012'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:55.539103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1248/2500] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:55:55 2022] DELETE /identity/v3/users/d543e04a4da34a159dbe3d66b9ddd012 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:55.737285 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6b9ce-17c1-441c-8bc4-486ef1b1c290 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:55.750980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6b9ce-17c1-441c-8bc4-486ef1b1c290 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:55.811135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e66649-8477-46e9-946e-4ad384d1cb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:55.811793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e66649-8477-46e9-946e-4ad384d1cb41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:55.812098 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e66649-8477-46e9-946e-4ad384d1cb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:55.812345 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e66649-8477-46e9-946e-4ad384d1cb41 admin admin] PATH_INFO: `/v3/projects/4eaa602c80cd44fbb764ce5a52c05a57` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:55.813526 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e66649-8477-46e9-946e-4ad384d1cb41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eaa602c80cd44fbb764ce5a52c05a57)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:55.815497 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e66649-8477-46e9-946e-4ad384d1cb41 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:55.865198 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-87e66649-8477-46e9-946e-4ad384d1cb41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eaa602c80cd44fbb764ce5a52c05a57', 'request_id': 'req-87e66649-8477-46e9-946e-4ad384d1cb41'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:55.873667 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-87e66649-8477-46e9-946e-4ad384d1cb41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eaa602c80cd44fbb764ce5a52c05a57', 'request_id': 'req-87e66649-8477-46e9-946e-4ad384d1cb41'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:55.881195 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-87e66649-8477-46e9-946e-4ad384d1cb41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eaa602c80cd44fbb764ce5a52c05a57', 'request_id': 'req-87e66649-8477-46e9-946e-4ad384d1cb41'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:55.894815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-87e66649-8477-46e9-946e-4ad384d1cb41 admin admin] The token cache is being invalidate because project 4eaa602c80cd44fbb764ce5a52c05a57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:55:55.895107 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-87e66649-8477-46e9-946e-4ad384d1cb41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:55.895751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-87e66649-8477-46e9-946e-4ad384d1cb41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:55.897284 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1253/2501] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:55:55 2022] DELETE /identity/v3/projects/4eaa602c80cd44fbb764ce5a52c05a57 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:55.901966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dd005-7011-4ba1-b259-374481eff2b8 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:55.903478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-eb7dd005-7011-4ba1-b259-374481eff2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:55.917320 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dd005-7011-4ba1-b259-374481eff2b8 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:55.983515 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e683427-c5d8-45d4-ad18-6a753ca45f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:55.984251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e683427-c5d8-45d4-ad18-6a753ca45f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:55.984529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e683427-c5d8-45d4-ad18-6a753ca45f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:55.984797 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e683427-c5d8-45d4-ad18-6a753ca45f08 admin admin] PATH_INFO: `/v3/users/fc04571f25874214a99973b96c1f4a5d` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:55.986058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e683427-c5d8-45d4-ad18-6a753ca45f08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc04571f25874214a99973b96c1f4a5d)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:55.988023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e683427-c5d8-45d4-ad18-6a753ca45f08 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:56.030259 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:56.040760 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5e683427-c5d8-45d4-ad18-6a753ca45f08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc04571f25874214a99973b96c1f4a5d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:56.041409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5e683427-c5d8-45d4-ad18-6a753ca45f08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc04571f25874214a99973b96c1f4a5d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:56.042007 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5e683427-c5d8-45d4-ad18-6a753ca45f08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc04571f25874214a99973b96c1f4a5d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:56.049556 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5e683427-c5d8-45d4-ad18-6a753ca45f08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc04571f25874214a99973b96c1f4a5d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:56.063367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5e683427-c5d8-45d4-ad18-6a753ca45f08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc04571f25874214a99973b96c1f4a5d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:56.080865 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1249/2502] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:55:55 2022] DELETE /identity/v3/users/fc04571f25874214a99973b96c1f4a5d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:56.278566 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e66649-8477-46e9-946e-4ad384d1cb41 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:56.280090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-87e66649-8477-46e9-946e-4ad384d1cb41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:56.299267 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e66649-8477-46e9-946e-4ad384d1cb41 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:56.369795 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c393f5-9ace-4ace-b5c9-fd79f6bc89d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:56.370537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c393f5-9ace-4ace-b5c9-fd79f6bc89d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:56.370912 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c393f5-9ace-4ace-b5c9-fd79f6bc89d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:56.371265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c393f5-9ace-4ace-b5c9-fd79f6bc89d8 admin admin] PATH_INFO: `/v3/projects/8e84888f58d74331b54c936cd8977b31` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:56.372855 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c393f5-9ace-4ace-b5c9-fd79f6bc89d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e84888f58d74331b54c936cd8977b31)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:56.376313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c393f5-9ace-4ace-b5c9-fd79f6bc89d8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:56.406480 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:56.437509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-66c393f5-9ace-4ace-b5c9-fd79f6bc89d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e84888f58d74331b54c936cd8977b31', 'request_id': 'req-66c393f5-9ace-4ace-b5c9-fd79f6bc89d8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:56.447634 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-66c393f5-9ace-4ace-b5c9-fd79f6bc89d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e84888f58d74331b54c936cd8977b31', 'request_id': 'req-66c393f5-9ace-4ace-b5c9-fd79f6bc89d8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:56.454890 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-66c393f5-9ace-4ace-b5c9-fd79f6bc89d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e84888f58d74331b54c936cd8977b31', 'request_id': 'req-66c393f5-9ace-4ace-b5c9-fd79f6bc89d8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:56.468339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-66c393f5-9ace-4ace-b5c9-fd79f6bc89d8 admin admin] The token cache is being invalidate because project 8e84888f58d74331b54c936cd8977b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:55:56.468687 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-66c393f5-9ace-4ace-b5c9-fd79f6bc89d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:56.469716 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-66c393f5-9ace-4ace-b5c9-fd79f6bc89d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:56.471955 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1254/2503] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:55:56 2022] DELETE /identity/v3/projects/8e84888f58d74331b54c936cd8977b31 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:56.478295 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdcead-905b-4294-aa57-e19cd64dcd5b None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:56.478811 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdcead-905b-4294-aa57-e19cd64dcd5b None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:56.478811 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdcead-905b-4294-aa57-e19cd64dcd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:56.501309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-53bdcead-905b-4294-aa57-e19cd64dcd5b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:56.502473 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-53bdcead-905b-4294-aa57-e19cd64dcd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:56.571157 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1250/2504] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:56.581111 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06840ceb-4d1e-4392-8c3e-cf242720cd96 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:56.581363 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06840ceb-4d1e-4392-8c3e-cf242720cd96 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:56.581642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06840ceb-4d1e-4392-8c3e-cf242720cd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:56.604418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-06840ceb-4d1e-4392-8c3e-cf242720cd96 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:56.606519 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-06840ceb-4d1e-4392-8c3e-cf242720cd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:56.621184 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1255/2505] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:56.625990 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bdcead-905b-4294-aa57-e19cd64dcd5b None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:56.636585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bdcead-905b-4294-aa57-e19cd64dcd5b None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:56.642945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19887df-75a5-4a19-8582-91aa4d68ff97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:56.643692 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19887df-75a5-4a19-8582-91aa4d68ff97 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:56.644104 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19887df-75a5-4a19-8582-91aa4d68ff97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:56.644466 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19887df-75a5-4a19-8582-91aa4d68ff97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:56.645396 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19887df-75a5-4a19-8582-91aa4d68ff97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:55:56.645813 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19887df-75a5-4a19-8582-91aa4d68ff97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:56.647571 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19887df-75a5-4a19-8582-91aa4d68ff97 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:56.662058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1251/2506] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:55:56.666745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06840ceb-4d1e-4392-8c3e-cf242720cd96 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:56.677416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06840ceb-4d1e-4392-8c3e-cf242720cd96 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:56.683360 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91217ab-85c8-4ad6-8e03-3759748d9341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:56.684034 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91217ab-85c8-4ad6-8e03-3759748d9341 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:56.684311 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91217ab-85c8-4ad6-8e03-3759748d9341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:56.684571 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91217ab-85c8-4ad6-8e03-3759748d9341 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:56.685240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91217ab-85c8-4ad6-8e03-3759748d9341 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:56.687295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91217ab-85c8-4ad6-8e03-3759748d9341 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:56.716971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1256/2507] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:55:56 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:55:56.722529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19887df-75a5-4a19-8582-91aa4d68ff97 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:56.732831 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19887df-75a5-4a19-8582-91aa4d68ff97 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:56.807359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eead98-f58b-49fe-b1d8-297d4f4f64ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:56.807359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eead98-f58b-49fe-b1d8-297d4f4f64ff admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:56.807359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eead98-f58b-49fe-b1d8-297d4f4f64ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:56.807359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eead98-f58b-49fe-b1d8-297d4f4f64ff admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:56.807923 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eead98-f58b-49fe-b1d8-297d4f4f64ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:56.807923 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eead98-f58b-49fe-b1d8-297d4f4f64ff admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:56.835591 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1252/2508] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:55:56 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:55:56.839785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91217ab-85c8-4ad6-8e03-3759748d9341 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:56.850535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91217ab-85c8-4ad6-8e03-3759748d9341 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:56.855377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa7118-3729-4630-9188-1c5b129168ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:56.855977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa7118-3729-4630-9188-1c5b129168ec admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:56.856276 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa7118-3729-4630-9188-1c5b129168ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:56.856537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa7118-3729-4630-9188-1c5b129168ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:56.857170 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa7118-3729-4630-9188-1c5b129168ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:56.858798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa7118-3729-4630-9188-1c5b129168ec admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:56.869572 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1257/2509] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:55:56.874536 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eead98-f58b-49fe-b1d8-297d4f4f64ff admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:56.889394 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eead98-f58b-49fe-b1d8-297d4f4f64ff admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:56.896741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87578f-cf0c-4171-a707-a1420b1d988c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:56.897380 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87578f-cf0c-4171-a707-a1420b1d988c admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:56.897670 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87578f-cf0c-4171-a707-a1420b1d988c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:56.898064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87578f-cf0c-4171-a707-a1420b1d988c admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:56.898722 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87578f-cf0c-4171-a707-a1420b1d988c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:56.900399 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87578f-cf0c-4171-a707-a1420b1d988c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:56.912264 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1253/2510] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:55:56.916739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa7118-3729-4630-9188-1c5b129168ec admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:56.931883 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa7118-3729-4630-9188-1c5b129168ec admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:56.940602 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e36a6d-3966-40d0-98cd-41c41a6e8967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:56.941306 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e36a6d-3966-40d0-98cd-41c41a6e8967 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:56.941588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e36a6d-3966-40d0-98cd-41c41a6e8967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:56.941846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e36a6d-3966-40d0-98cd-41c41a6e8967 admin admin] PATH_INFO: `/v3/projects/e2de9070fb6b4ed69534db387b0d4d05/users/d407be54b367490b919e78953c7645d6/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:56.959260 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e36a6d-3966-40d0-98cd-41c41a6e8967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2de9070fb6b4ed69534db387b0d4d05, user_id=d407be54b367490b919e78953c7645d6, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:56.961937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e36a6d-3966-40d0-98cd-41c41a6e8967 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:56.976706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1258/2511] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:55:56 2022] PUT /identity/v3/projects/e2de9070fb6b4ed69534db387b0d4d05/users/d407be54b367490b919e78953c7645d6/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:55:56.989217 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5a6ae-7362-47d3-b1d9-2ee44787e913 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:56.989615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5a6ae-7362-47d3-b1d9-2ee44787e913 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:56.990039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5a6ae-7362-47d3-b1d9-2ee44787e913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:57.032625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-d2e5a6ae-7362-47d3-b1d9-2ee44787e913 None None] MFA Rules not processed for user `d407be54b367490b919e78953c7645d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:57.033905 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d2e5a6ae-7362-47d3-b1d9-2ee44787e913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:57.043029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:57.107693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1254/2512] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:57.112649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e36a6d-3966-40d0-98cd-41c41a6e8967 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:57.178617 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e36a6d-3966-40d0-98cd-41c41a6e8967 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:57.183038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bef959-fbbd-464d-996f-ead2ddf45d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:57.183647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bef959-fbbd-464d-996f-ead2ddf45d08 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:57.183925 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bef959-fbbd-464d-996f-ead2ddf45d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:57.184221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bef959-fbbd-464d-996f-ead2ddf45d08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:57.184856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bef959-fbbd-464d-996f-ead2ddf45d08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:57.186778 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bef959-fbbd-464d-996f-ead2ddf45d08 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:57.215390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1259/2513] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:55:57 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:55:57.219872 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5a6ae-7362-47d3-b1d9-2ee44787e913 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:57.231755 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5a6ae-7362-47d3-b1d9-2ee44787e913 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:57.299010 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1ebbd-7e29-487a-ab8f-c098e39c7282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:57.299624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1ebbd-7e29-487a-ab8f-c098e39c7282 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:57.299907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1ebbd-7e29-487a-ab8f-c098e39c7282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:57.300202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1ebbd-7e29-487a-ab8f-c098e39c7282 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:57.301219 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1ebbd-7e29-487a-ab8f-c098e39c7282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:57.303149 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1ebbd-7e29-487a-ab8f-c098e39c7282 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:57.328935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1255/2514] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:55:57 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:55:57.333994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bef959-fbbd-464d-996f-ead2ddf45d08 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:57.347067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bef959-fbbd-464d-996f-ead2ddf45d08 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:57.352349 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e4d9a-6559-4459-a3c2-5e5791cc3db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:57.352956 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e4d9a-6559-4459-a3c2-5e5791cc3db1 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:57.353231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e4d9a-6559-4459-a3c2-5e5791cc3db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:57.353499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e4d9a-6559-4459-a3c2-5e5791cc3db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:57.354138 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e4d9a-6559-4459-a3c2-5e5791cc3db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:57.355744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e4d9a-6559-4459-a3c2-5e5791cc3db1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:57.368928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1260/2515] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:55:57.373987 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1ebbd-7e29-487a-ab8f-c098e39c7282 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:57.385162 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1ebbd-7e29-487a-ab8f-c098e39c7282 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:57.390372 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3751b4b-2596-4249-b06b-7ab95d865f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:57.391055 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3751b4b-2596-4249-b06b-7ab95d865f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:57.391333 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3751b4b-2596-4249-b06b-7ab95d865f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:57.391601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3751b4b-2596-4249-b06b-7ab95d865f0a admin admin] PATH_INFO: `/v3/projects/6f210323ecff44bebbaa643b1c053ccf/users/3688ae274b95459b80744f0df22b6ff0/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:57.407835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3751b4b-2596-4249-b06b-7ab95d865f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f210323ecff44bebbaa643b1c053ccf, user_id=3688ae274b95459b80744f0df22b6ff0, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:57.410542 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3751b4b-2596-4249-b06b-7ab95d865f0a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:57.426541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1256/2516] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:55:57 2022] PUT /identity/v3/projects/6f210323ecff44bebbaa643b1c053ccf/users/3688ae274b95459b80744f0df22b6ff0/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:57.431351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e4d9a-6559-4459-a3c2-5e5791cc3db1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:57.442720 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e4d9a-6559-4459-a3c2-5e5791cc3db1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:57.462857 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:57.516425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28944556-2609-40cd-88ea-2ee69fa82d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:57.517067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28944556-2609-40cd-88ea-2ee69fa82d94 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:57.517341 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28944556-2609-40cd-88ea-2ee69fa82d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:57.517605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28944556-2609-40cd-88ea-2ee69fa82d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:57.518256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28944556-2609-40cd-88ea-2ee69fa82d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:57.519882 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28944556-2609-40cd-88ea-2ee69fa82d94 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:57.532904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1261/2517] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:55:57.537397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3751b4b-2596-4249-b06b-7ab95d865f0a admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:57.554471 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3751b4b-2596-4249-b06b-7ab95d865f0a admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:57.562611 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20793d54-f4c3-422e-9c82-7bad91b63190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:57.563272 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20793d54-f4c3-422e-9c82-7bad91b63190 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:57.563544 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20793d54-f4c3-422e-9c82-7bad91b63190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:57.563790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20793d54-f4c3-422e-9c82-7bad91b63190 admin admin] PATH_INFO: `/v3/projects/6f210323ecff44bebbaa643b1c053ccf/users/3688ae274b95459b80744f0df22b6ff0/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:57.565921 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20793d54-f4c3-422e-9c82-7bad91b63190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f210323ecff44bebbaa643b1c053ccf, user_id=3688ae274b95459b80744f0df22b6ff0, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:57.568595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20793d54-f4c3-422e-9c82-7bad91b63190 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:57.580777 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1257/2518] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:55:57 2022] PUT /identity/v3/projects/6f210323ecff44bebbaa643b1c053ccf/users/3688ae274b95459b80744f0df22b6ff0/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:55:57.592489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d4106-94cd-4c42-b1e3-c1973653d30e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:57.593359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d4106-94cd-4c42-b1e3-c1973653d30e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:57.593666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d4106-94cd-4c42-b1e3-c1973653d30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:57.624667 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-eb8d4106-94cd-4c42-b1e3-c1973653d30e None None] MFA Rules not processed for user `3688ae274b95459b80744f0df22b6ff0`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:57.625916 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-eb8d4106-94cd-4c42-b1e3-c1973653d30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:57.705269 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1262/2519] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:57.719852 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20793d54-f4c3-422e-9c82-7bad91b63190 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:57.721832 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-20793d54-f4c3-422e-9c82-7bad91b63190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:57.785962 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20793d54-f4c3-422e-9c82-7bad91b63190 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:57.789680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e25bcf-4a17-4ed3-8812-cff6734b133e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:57.790299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e25bcf-4a17-4ed3-8812-cff6734b133e service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:57.790572 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e25bcf-4a17-4ed3-8812-cff6734b133e service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:57.790840 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e25bcf-4a17-4ed3-8812-cff6734b133e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:57.801335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e25bcf-4a17-4ed3-8812-cff6734b133e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:57.802781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e25bcf-4a17-4ed3-8812-cff6734b133e service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:57.812825 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1258/2520] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:55:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:55:58.284685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d4106-94cd-4c42-b1e3-c1973653d30e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:58.361597 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d4106-94cd-4c42-b1e3-c1973653d30e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:58.370012 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe712dd-c3b2-4b59-ac36-e04e2074bdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:58.370967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe712dd-c3b2-4b59-ac36-e04e2074bdb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:58.371274 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe712dd-c3b2-4b59-ac36-e04e2074bdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:58.371615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe712dd-c3b2-4b59-ac36-e04e2074bdb7 admin admin] PATH_INFO: `/v3/users/d407be54b367490b919e78953c7645d6` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:58.373533 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe712dd-c3b2-4b59-ac36-e04e2074bdb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d407be54b367490b919e78953c7645d6)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:58.375512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe712dd-c3b2-4b59-ac36-e04e2074bdb7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:58.427798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:58.434217 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9fe712dd-c3b2-4b59-ac36-e04e2074bdb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd407be54b367490b919e78953c7645d6'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:58.442978 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9fe712dd-c3b2-4b59-ac36-e04e2074bdb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd407be54b367490b919e78953c7645d6'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:58.453104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9fe712dd-c3b2-4b59-ac36-e04e2074bdb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd407be54b367490b919e78953c7645d6'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:58.453891 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9fe712dd-c3b2-4b59-ac36-e04e2074bdb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd407be54b367490b919e78953c7645d6'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:58.468127 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9fe712dd-c3b2-4b59-ac36-e04e2074bdb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd407be54b367490b919e78953c7645d6'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:58.474878 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1263/2521] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:55:58 2022] DELETE /identity/v3/users/d407be54b367490b919e78953c7645d6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:58.479728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5f4d2-655e-48e5-8f57-ea89731b8aea None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:58.480095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5f4d2-655e-48e5-8f57-ea89731b8aea None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:58.480263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5f4d2-655e-48e5-8f57-ea89731b8aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:58.488931 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:58.500629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-7aa5f4d2-655e-48e5-8f57-ea89731b8aea None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:58.501883 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7aa5f4d2-655e-48e5-8f57-ea89731b8aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:58.572442 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1259/2522] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:58.583782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe712dd-c3b2-4b59-ac36-e04e2074bdb7 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:58.585360 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9fe712dd-c3b2-4b59-ac36-e04e2074bdb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:58.650626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe712dd-c3b2-4b59-ac36-e04e2074bdb7 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:58.655190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53a9be-93ec-4514-b872-6255c387298f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:58.655805 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53a9be-93ec-4514-b872-6255c387298f service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:58.656113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53a9be-93ec-4514-b872-6255c387298f service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:58.656429 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53a9be-93ec-4514-b872-6255c387298f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:58.669094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53a9be-93ec-4514-b872-6255c387298f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:58.671253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53a9be-93ec-4514-b872-6255c387298f service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:58.683164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1264/2523] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:55:58.990736 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5f4d2-655e-48e5-8f57-ea89731b8aea None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:58.993090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7aa5f4d2-655e-48e5-8f57-ea89731b8aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:59.009248 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5f4d2-655e-48e5-8f57-ea89731b8aea None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:59.016530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3327476-68ca-4fcd-987f-9219e13664f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:59.017223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3327476-68ca-4fcd-987f-9219e13664f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:59.017513 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3327476-68ca-4fcd-987f-9219e13664f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:59.017805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3327476-68ca-4fcd-987f-9219e13664f1 admin admin] PATH_INFO: `/v3/projects/e2de9070fb6b4ed69534db387b0d4d05` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:59.019244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3327476-68ca-4fcd-987f-9219e13664f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2de9070fb6b4ed69534db387b0d4d05)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:59.021137 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3327476-68ca-4fcd-987f-9219e13664f1 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:59.077402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b3327476-68ca-4fcd-987f-9219e13664f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2de9070fb6b4ed69534db387b0d4d05', 'request_id': 'req-b3327476-68ca-4fcd-987f-9219e13664f1'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:59.083250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b3327476-68ca-4fcd-987f-9219e13664f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2de9070fb6b4ed69534db387b0d4d05', 'request_id': 'req-b3327476-68ca-4fcd-987f-9219e13664f1'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:59.091428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b3327476-68ca-4fcd-987f-9219e13664f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2de9070fb6b4ed69534db387b0d4d05', 'request_id': 'req-b3327476-68ca-4fcd-987f-9219e13664f1'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:59.111134 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b3327476-68ca-4fcd-987f-9219e13664f1 admin admin] The token cache is being invalidate because project e2de9070fb6b4ed69534db387b0d4d05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:55:59.111379 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b3327476-68ca-4fcd-987f-9219e13664f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:59.112150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b3327476-68ca-4fcd-987f-9219e13664f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:59.113688 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1260/2524] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:55:58 2022] DELETE /identity/v3/projects/e2de9070fb6b4ed69534db387b0d4d05 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:59.118598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53a9be-93ec-4514-b872-6255c387298f service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:59.131057 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53a9be-93ec-4514-b872-6255c387298f service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:59.200680 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ba8ae-49c5-455b-af16-de9814522a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:59.201309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ba8ae-49c5-455b-af16-de9814522a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:59.201583 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ba8ae-49c5-455b-af16-de9814522a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:59.201926 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ba8ae-49c5-455b-af16-de9814522a53 admin admin] PATH_INFO: `/v3/users/3688ae274b95459b80744f0df22b6ff0` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:59.203095 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ba8ae-49c5-455b-af16-de9814522a53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3688ae274b95459b80744f0df22b6ff0)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:59.205047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ba8ae-49c5-455b-af16-de9814522a53 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:59.256277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-215ba8ae-49c5-455b-af16-de9814522a53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3688ae274b95459b80744f0df22b6ff0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:59.263620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-215ba8ae-49c5-455b-af16-de9814522a53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3688ae274b95459b80744f0df22b6ff0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:59.273156 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-215ba8ae-49c5-455b-af16-de9814522a53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3688ae274b95459b80744f0df22b6ff0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:59.273840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-215ba8ae-49c5-455b-af16-de9814522a53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3688ae274b95459b80744f0df22b6ff0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:59.275309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:59.294659 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-215ba8ae-49c5-455b-af16-de9814522a53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3688ae274b95459b80744f0df22b6ff0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:59.305414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1265/2525] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:55:59 2022] DELETE /identity/v3/users/3688ae274b95459b80744f0df22b6ff0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:59.613806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3327476-68ca-4fcd-987f-9219e13664f1 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:59.615464 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b3327476-68ca-4fcd-987f-9219e13664f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:59.632344 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3327476-68ca-4fcd-987f-9219e13664f1 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:59.691762 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8711f-78e3-438e-84c2-3c78b85db9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:59.692433 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8711f-78e3-438e-84c2-3c78b85db9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:59.692706 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8711f-78e3-438e-84c2-3c78b85db9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:59.692971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8711f-78e3-438e-84c2-3c78b85db9b3 admin admin] PATH_INFO: `/v3/projects/6f210323ecff44bebbaa643b1c053ccf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:59.694169 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d8711f-78e3-438e-84c2-3c78b85db9b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f210323ecff44bebbaa643b1c053ccf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:59.696124 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d8711f-78e3-438e-84c2-3c78b85db9b3 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:59.734153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:59.751291 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-37d8711f-78e3-438e-84c2-3c78b85db9b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f210323ecff44bebbaa643b1c053ccf', 'request_id': 'req-37d8711f-78e3-438e-84c2-3c78b85db9b3'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:59.757366 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-37d8711f-78e3-438e-84c2-3c78b85db9b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f210323ecff44bebbaa643b1c053ccf', 'request_id': 'req-37d8711f-78e3-438e-84c2-3c78b85db9b3'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:59.763935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-37d8711f-78e3-438e-84c2-3c78b85db9b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f210323ecff44bebbaa643b1c053ccf', 'request_id': 'req-37d8711f-78e3-438e-84c2-3c78b85db9b3'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:59.779655 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-37d8711f-78e3-438e-84c2-3c78b85db9b3 admin admin] The token cache is being invalidate because project 6f210323ecff44bebbaa643b1c053ccf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:55:59.779958 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-37d8711f-78e3-438e-84c2-3c78b85db9b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:59.780748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-37d8711f-78e3-438e-84c2-3c78b85db9b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:55:59.782427 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1261/2526] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:55:59 2022] DELETE /identity/v3/projects/6f210323ecff44bebbaa643b1c053ccf => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:55:59.788271 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca044f-4f00-4bc1-a005-578b4272d7a2 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:59.788639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca044f-4f00-4bc1-a005-578b4272d7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:59.789026 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca044f-4f00-4bc1-a005-578b4272d7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:59.811895 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-99ca044f-4f00-4bc1-a005-578b4272d7a2 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:59.813134 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-99ca044f-4f00-4bc1-a005-578b4272d7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:59.876482 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1266/2527] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:59.884900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee00964-ec3f-49fe-9199-ff6919592b95 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:59.885190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee00964-ec3f-49fe-9199-ff6919592b95 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:59.885496 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee00964-ec3f-49fe-9199-ff6919592b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:59.906329 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-aee00964-ec3f-49fe-9199-ff6919592b95 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:55:59.907573 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-aee00964-ec3f-49fe-9199-ff6919592b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:59.916372 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1262/2528] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:55:59.921463 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca044f-4f00-4bc1-a005-578b4272d7a2 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:59.932333 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca044f-4f00-4bc1-a005-578b4272d7a2 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:59.937886 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66f268-a465-40f5-aaa6-2c3d7f2f332e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:59.938576 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66f268-a465-40f5-aaa6-2c3d7f2f332e admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:59.938841 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66f268-a465-40f5-aaa6-2c3d7f2f332e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:59.939203 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66f268-a465-40f5-aaa6-2c3d7f2f332e admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:59.940243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc66f268-a465-40f5-aaa6-2c3d7f2f332e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:55:59.940643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc66f268-a465-40f5-aaa6-2c3d7f2f332e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:59.942915 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc66f268-a465-40f5-aaa6-2c3d7f2f332e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:59.958760 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1267/2529] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:55:59.963371 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee00964-ec3f-49fe-9199-ff6919592b95 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:59.973795 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee00964-ec3f-49fe-9199-ff6919592b95 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:59.979773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0ddbe-0e4b-42c9-895b-0b0d759c0c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:59.980440 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0ddbe-0e4b-42c9-895b-0b0d759c0c4c admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:59.980711 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0ddbe-0e4b-42c9-895b-0b0d759c0c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:59.980968 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0ddbe-0e4b-42c9-895b-0b0d759c0c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:59.981591 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0ddbe-0e4b-42c9-895b-0b0d759c0c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:59.983551 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0ddbe-0e4b-42c9-895b-0b0d759c0c4c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:00.015299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1263/2530] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:55:59 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:00.020678 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66f268-a465-40f5-aaa6-2c3d7f2f332e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:00.032885 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66f268-a465-40f5-aaa6-2c3d7f2f332e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:00.096856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd79c0e-0a86-47bc-a79f-b7dff967fe98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:00.097477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd79c0e-0a86-47bc-a79f-b7dff967fe98 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:00.097749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd79c0e-0a86-47bc-a79f-b7dff967fe98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:00.098014 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd79c0e-0a86-47bc-a79f-b7dff967fe98 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:00.098762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd79c0e-0a86-47bc-a79f-b7dff967fe98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:00.100756 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd79c0e-0a86-47bc-a79f-b7dff967fe98 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:00.128191 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1268/2531] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:00 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:00.133836 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0ddbe-0e4b-42c9-895b-0b0d759c0c4c admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:00.147104 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0ddbe-0e4b-42c9-895b-0b0d759c0c4c admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:00.152649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dfb55-b0a1-4227-b689-d319c8f8e8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:00.153262 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dfb55-b0a1-4227-b689-d319c8f8e8bb admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:00.153537 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dfb55-b0a1-4227-b689-d319c8f8e8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:00.153799 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dfb55-b0a1-4227-b689-d319c8f8e8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:00.154441 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8dfb55-b0a1-4227-b689-d319c8f8e8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:00.156151 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8dfb55-b0a1-4227-b689-d319c8f8e8bb admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:00.166959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1264/2532] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:00.171804 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd79c0e-0a86-47bc-a79f-b7dff967fe98 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:00.183247 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd79c0e-0a86-47bc-a79f-b7dff967fe98 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:00.190338 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e10338-c5de-4693-8b37-23bb35da5129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:00.190941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e10338-c5de-4693-8b37-23bb35da5129 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:00.191214 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e10338-c5de-4693-8b37-23bb35da5129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:00.191481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e10338-c5de-4693-8b37-23bb35da5129 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:00.192170 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e10338-c5de-4693-8b37-23bb35da5129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:00.193804 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e10338-c5de-4693-8b37-23bb35da5129 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:00.206553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1269/2533] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:00.212672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dfb55-b0a1-4227-b689-d319c8f8e8bb admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:00.223165 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dfb55-b0a1-4227-b689-d319c8f8e8bb admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:00.228469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90caf2d4-04d2-4b96-9b5b-5066a39ea0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:00.229119 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90caf2d4-04d2-4b96-9b5b-5066a39ea0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:00.229416 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90caf2d4-04d2-4b96-9b5b-5066a39ea0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:00.229683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90caf2d4-04d2-4b96-9b5b-5066a39ea0d2 admin admin] PATH_INFO: `/v3/projects/0af36aa833bc49caab14f40ca7ad3fe2/users/1d47f73ebf994e8fa7bad70de41cc8ba/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:00.245933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90caf2d4-04d2-4b96-9b5b-5066a39ea0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af36aa833bc49caab14f40ca7ad3fe2, user_id=1d47f73ebf994e8fa7bad70de41cc8ba, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:00.249176 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90caf2d4-04d2-4b96-9b5b-5066a39ea0d2 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:00.265351 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1265/2534] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:00 2022] PUT /identity/v3/projects/0af36aa833bc49caab14f40ca7ad3fe2/users/1d47f73ebf994e8fa7bad70de41cc8ba/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:00.276880 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b99a96-a3ef-442d-b137-b2609d7080e4 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:00.277160 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b99a96-a3ef-442d-b137-b2609d7080e4 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:00.277478 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b99a96-a3ef-442d-b137-b2609d7080e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:00.311298 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-58b99a96-a3ef-442d-b137-b2609d7080e4 None None] MFA Rules not processed for user `1d47f73ebf994e8fa7bad70de41cc8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:00.312596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-58b99a96-a3ef-442d-b137-b2609d7080e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:00.350148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:00.390977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1270/2535] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:00.397101 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90caf2d4-04d2-4b96-9b5b-5066a39ea0d2 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:00.479064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90caf2d4-04d2-4b96-9b5b-5066a39ea0d2 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:00.484858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ec59c-358c-4cb0-8118-d72f6d58f224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:00.485468 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ec59c-358c-4cb0-8118-d72f6d58f224 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:00.485742 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ec59c-358c-4cb0-8118-d72f6d58f224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:00.486011 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ec59c-358c-4cb0-8118-d72f6d58f224 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:00.486646 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ec59c-358c-4cb0-8118-d72f6d58f224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:00.488590 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ec59c-358c-4cb0-8118-d72f6d58f224 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:00.521111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1266/2536] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:00 2022] POST /identity/v3/projects => generated 391 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:00.525953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b99a96-a3ef-442d-b137-b2609d7080e4 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:00.537914 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b99a96-a3ef-442d-b137-b2609d7080e4 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:00.597662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3767e-2978-4b48-aa79-a717b6c11cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:00.598278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3767e-2978-4b48-aa79-a717b6c11cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:00.598549 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3767e-2978-4b48-aa79-a717b6c11cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:00.598811 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3767e-2978-4b48-aa79-a717b6c11cf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:00.599451 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3767e-2978-4b48-aa79-a717b6c11cf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:00.601411 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3767e-2978-4b48-aa79-a717b6c11cf0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:00.626036 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1271/2537] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:00 2022] POST /identity/v3/users => generated 302 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:56:00.630285 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ec59c-358c-4cb0-8118-d72f6d58f224 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:00.641569 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ec59c-358c-4cb0-8118-d72f6d58f224 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:00.646876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378054c-86b7-44ab-8c94-221270e35262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:00.647634 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378054c-86b7-44ab-8c94-221270e35262 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:00.647910 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378054c-86b7-44ab-8c94-221270e35262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:00.648214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378054c-86b7-44ab-8c94-221270e35262 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:00.648856 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378054c-86b7-44ab-8c94-221270e35262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:00.650456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378054c-86b7-44ab-8c94-221270e35262 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:00.661524 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1267/2538] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:00.665993 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3767e-2978-4b48-aa79-a717b6c11cf0 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:00.677771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3767e-2978-4b48-aa79-a717b6c11cf0 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:00.686614 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da46587-cc24-436a-90ce-8ce142fbe77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:00.687287 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da46587-cc24-436a-90ce-8ce142fbe77b admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:00.687564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da46587-cc24-436a-90ce-8ce142fbe77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:00.687828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da46587-cc24-436a-90ce-8ce142fbe77b admin admin] PATH_INFO: `/v3/projects/edfbd40698c144d9b98cdc11b440d6a4/users/018b4d84c1104154b366e689f0907ff3/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:00.706157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da46587-cc24-436a-90ce-8ce142fbe77b admin admin] RBAC: Authorizing `identity:create_grant(project_id=edfbd40698c144d9b98cdc11b440d6a4, user_id=018b4d84c1104154b366e689f0907ff3, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:00.708900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da46587-cc24-436a-90ce-8ce142fbe77b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:00.730256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1272/2539] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:00 2022] PUT /identity/v3/projects/edfbd40698c144d9b98cdc11b440d6a4/users/018b4d84c1104154b366e689f0907ff3/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:00.735620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378054c-86b7-44ab-8c94-221270e35262 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:00.746162 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378054c-86b7-44ab-8c94-221270e35262 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:00.774236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:00.807411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448fc84-29df-4e63-803e-88d2e2575476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:00.808044 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448fc84-29df-4e63-803e-88d2e2575476 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:00.808325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448fc84-29df-4e63-803e-88d2e2575476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:00.808616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448fc84-29df-4e63-803e-88d2e2575476 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:00.809254 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448fc84-29df-4e63-803e-88d2e2575476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:00.810849 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448fc84-29df-4e63-803e-88d2e2575476 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:00.821597 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1268/2540] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:56:00.825916 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da46587-cc24-436a-90ce-8ce142fbe77b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:00.837475 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da46587-cc24-436a-90ce-8ce142fbe77b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:00.843612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c898d43-e421-468d-871d-5d6bc3a7196e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:00.844411 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c898d43-e421-468d-871d-5d6bc3a7196e admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:00.844683 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c898d43-e421-468d-871d-5d6bc3a7196e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:00.844949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c898d43-e421-468d-871d-5d6bc3a7196e admin admin] PATH_INFO: `/v3/projects/edfbd40698c144d9b98cdc11b440d6a4/users/018b4d84c1104154b366e689f0907ff3/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:00.846914 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c898d43-e421-468d-871d-5d6bc3a7196e admin admin] RBAC: Authorizing `identity:create_grant(project_id=edfbd40698c144d9b98cdc11b440d6a4, user_id=018b4d84c1104154b366e689f0907ff3, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:00.849617 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c898d43-e421-468d-871d-5d6bc3a7196e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:00.860427 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1273/2541] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:00 2022] PUT /identity/v3/projects/edfbd40698c144d9b98cdc11b440d6a4/users/018b4d84c1104154b366e689f0907ff3/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:00.871540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d7857-05cb-4fa8-b631-b1fc9b063bac None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:00.871892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d7857-05cb-4fa8-b631-b1fc9b063bac None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:00.872308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d7857-05cb-4fa8-b631-b1fc9b063bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:00.908209 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a37d7857-05cb-4fa8-b631-b1fc9b063bac None None] MFA Rules not processed for user `018b4d84c1104154b366e689f0907ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:00.909427 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a37d7857-05cb-4fa8-b631-b1fc9b063bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:00.986856 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1269/2542] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:00 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:00.998607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c898d43-e421-468d-871d-5d6bc3a7196e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:01.001560 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6c898d43-e421-468d-871d-5d6bc3a7196e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:01.074114 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c898d43-e421-468d-871d-5d6bc3a7196e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:01.079955 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e847c-b8c0-4759-add8-1eb0da5a311a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:01.080701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e847c-b8c0-4759-add8-1eb0da5a311a service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:01.080972 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e847c-b8c0-4759-add8-1eb0da5a311a service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:01.081234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e847c-b8c0-4759-add8-1eb0da5a311a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:01.091067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e847c-b8c0-4759-add8-1eb0da5a311a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:01.092489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e847c-b8c0-4759-add8-1eb0da5a311a service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:01.104537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1274/2543] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:00 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:01.159503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d7857-05cb-4fa8-b631-b1fc9b063bac None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:01.161064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a37d7857-05cb-4fa8-b631-b1fc9b063bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:01.174973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d7857-05cb-4fa8-b631-b1fc9b063bac None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:01.179314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c781726-3d58-4b7e-ab51-0f38563dc52b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:01.179940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c781726-3d58-4b7e-ab51-0f38563dc52b service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:01.180252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c781726-3d58-4b7e-ab51-0f38563dc52b service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:01.180528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c781726-3d58-4b7e-ab51-0f38563dc52b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:01.240288 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c781726-3d58-4b7e-ab51-0f38563dc52b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:01.241734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c781726-3d58-4b7e-ab51-0f38563dc52b service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:01.251881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1270/2544] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:01 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:05.785801 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e847c-b8c0-4759-add8-1eb0da5a311a service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:05.821588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:05.856987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e847c-b8c0-4759-add8-1eb0da5a311a service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:05.861788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a91189-1803-4da1-be75-928384077dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:05.862362 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a91189-1803-4da1-be75-928384077dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:05.862634 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a91189-1803-4da1-be75-928384077dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:05.862898 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a91189-1803-4da1-be75-928384077dc4 admin admin] PATH_INFO: `/v3/users/1d47f73ebf994e8fa7bad70de41cc8ba` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:05.864185 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a91189-1803-4da1-be75-928384077dc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d47f73ebf994e8fa7bad70de41cc8ba)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:05.866095 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a91189-1803-4da1-be75-928384077dc4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:05.915897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-70a91189-1803-4da1-be75-928384077dc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d47f73ebf994e8fa7bad70de41cc8ba'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:05.924051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-70a91189-1803-4da1-be75-928384077dc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d47f73ebf994e8fa7bad70de41cc8ba'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:05.933356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-70a91189-1803-4da1-be75-928384077dc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d47f73ebf994e8fa7bad70de41cc8ba'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:05.933990 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-70a91189-1803-4da1-be75-928384077dc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d47f73ebf994e8fa7bad70de41cc8ba'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:05.947293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-70a91189-1803-4da1-be75-928384077dc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d47f73ebf994e8fa7bad70de41cc8ba'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:05.954279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1275/2545] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:05 2022] DELETE /identity/v3/users/1d47f73ebf994e8fa7bad70de41cc8ba => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:05.959162 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638874bb-97bf-41bf-bccc-701965045a2a None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:05.959660 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638874bb-97bf-41bf-bccc-701965045a2a None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:05.959865 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638874bb-97bf-41bf-bccc-701965045a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:05.987564 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-638874bb-97bf-41bf-bccc-701965045a2a None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:05.989073 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-638874bb-97bf-41bf-bccc-701965045a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:06.043531 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:06.064340 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1271/2546] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:06.071550 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a91189-1803-4da1-be75-928384077dc4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:06.073078 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-70a91189-1803-4da1-be75-928384077dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:06.133295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a91189-1803-4da1-be75-928384077dc4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:06.137325 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e9e54-317a-4410-aef8-72afe7490ce8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:06.137997 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e9e54-317a-4410-aef8-72afe7490ce8 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:06.138368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e9e54-317a-4410-aef8-72afe7490ce8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:06.138797 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e9e54-317a-4410-aef8-72afe7490ce8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:06.148704 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e9e54-317a-4410-aef8-72afe7490ce8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:06.150124 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e9e54-317a-4410-aef8-72afe7490ce8 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:06.161993 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1276/2547] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:06.457995 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638874bb-97bf-41bf-bccc-701965045a2a None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:06.460072 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-638874bb-97bf-41bf-bccc-701965045a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:06.473609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638874bb-97bf-41bf-bccc-701965045a2a None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:06.478482 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903a3b7-4503-46a2-95cc-7c5e15e893ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:06.479206 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903a3b7-4503-46a2-95cc-7c5e15e893ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:06.479551 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903a3b7-4503-46a2-95cc-7c5e15e893ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:06.479902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903a3b7-4503-46a2-95cc-7c5e15e893ae admin admin] PATH_INFO: `/v3/projects/0af36aa833bc49caab14f40ca7ad3fe2` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:06.481135 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903a3b7-4503-46a2-95cc-7c5e15e893ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=0af36aa833bc49caab14f40ca7ad3fe2)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:06.483216 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903a3b7-4503-46a2-95cc-7c5e15e893ae admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:06.553359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f903a3b7-4503-46a2-95cc-7c5e15e893ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0af36aa833bc49caab14f40ca7ad3fe2', 'request_id': 'req-f903a3b7-4503-46a2-95cc-7c5e15e893ae'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:06.559279 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f903a3b7-4503-46a2-95cc-7c5e15e893ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0af36aa833bc49caab14f40ca7ad3fe2', 'request_id': 'req-f903a3b7-4503-46a2-95cc-7c5e15e893ae'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:06.566990 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f903a3b7-4503-46a2-95cc-7c5e15e893ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0af36aa833bc49caab14f40ca7ad3fe2', 'request_id': 'req-f903a3b7-4503-46a2-95cc-7c5e15e893ae'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:06.585880 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f903a3b7-4503-46a2-95cc-7c5e15e893ae admin admin] The token cache is being invalidate because project 0af36aa833bc49caab14f40ca7ad3fe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:06.586174 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f903a3b7-4503-46a2-95cc-7c5e15e893ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:06.586948 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f903a3b7-4503-46a2-95cc-7c5e15e893ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:06.588595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1272/2548] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:06 2022] DELETE /identity/v3/projects/0af36aa833bc49caab14f40ca7ad3fe2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:06.592789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e9e54-317a-4410-aef8-72afe7490ce8 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:06.608443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e9e54-317a-4410-aef8-72afe7490ce8 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:06.662788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:06.674532 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17abde7-ec1b-446b-a9e9-92f73f3773e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:06.675523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17abde7-ec1b-446b-a9e9-92f73f3773e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:06.675802 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17abde7-ec1b-446b-a9e9-92f73f3773e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:06.676236 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17abde7-ec1b-446b-a9e9-92f73f3773e3 admin admin] PATH_INFO: `/v3/users/018b4d84c1104154b366e689f0907ff3` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:06.677685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17abde7-ec1b-446b-a9e9-92f73f3773e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=018b4d84c1104154b366e689f0907ff3)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:06.679828 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17abde7-ec1b-446b-a9e9-92f73f3773e3 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:06.732625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b17abde7-ec1b-446b-a9e9-92f73f3773e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '018b4d84c1104154b366e689f0907ff3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:06.740299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b17abde7-ec1b-446b-a9e9-92f73f3773e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '018b4d84c1104154b366e689f0907ff3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:06.750555 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b17abde7-ec1b-446b-a9e9-92f73f3773e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '018b4d84c1104154b366e689f0907ff3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:06.751271 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b17abde7-ec1b-446b-a9e9-92f73f3773e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '018b4d84c1104154b366e689f0907ff3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:06.765906 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b17abde7-ec1b-446b-a9e9-92f73f3773e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '018b4d84c1104154b366e689f0907ff3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:06.771957 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1277/2549] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:06 2022] DELETE /identity/v3/users/018b4d84c1104154b366e689f0907ff3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:07.063958 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903a3b7-4503-46a2-95cc-7c5e15e893ae admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:07.065485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f903a3b7-4503-46a2-95cc-7c5e15e893ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:07.085022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903a3b7-4503-46a2-95cc-7c5e15e893ae admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:07.151467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca746bd-47a0-4fe1-99df-84ee97596490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:07.152199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca746bd-47a0-4fe1-99df-84ee97596490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:07.152480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca746bd-47a0-4fe1-99df-84ee97596490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:07.152749 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca746bd-47a0-4fe1-99df-84ee97596490 admin admin] PATH_INFO: `/v3/projects/edfbd40698c144d9b98cdc11b440d6a4` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:07.153889 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca746bd-47a0-4fe1-99df-84ee97596490 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edfbd40698c144d9b98cdc11b440d6a4)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:07.155849 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca746bd-47a0-4fe1-99df-84ee97596490 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:07.205243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4ca746bd-47a0-4fe1-99df-84ee97596490 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edfbd40698c144d9b98cdc11b440d6a4', 'request_id': 'req-4ca746bd-47a0-4fe1-99df-84ee97596490'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:07.211206 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4ca746bd-47a0-4fe1-99df-84ee97596490 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edfbd40698c144d9b98cdc11b440d6a4', 'request_id': 'req-4ca746bd-47a0-4fe1-99df-84ee97596490'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:07.217791 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4ca746bd-47a0-4fe1-99df-84ee97596490 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edfbd40698c144d9b98cdc11b440d6a4', 'request_id': 'req-4ca746bd-47a0-4fe1-99df-84ee97596490'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:07.233552 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4ca746bd-47a0-4fe1-99df-84ee97596490 admin admin] The token cache is being invalidate because project edfbd40698c144d9b98cdc11b440d6a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:07.233930 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4ca746bd-47a0-4fe1-99df-84ee97596490 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:07.234668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4ca746bd-47a0-4fe1-99df-84ee97596490 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:07.236676 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1273/2550] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:07 2022] DELETE /identity/v3/projects/edfbd40698c144d9b98cdc11b440d6a4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:07.242090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345a6d5-ccc3-40bc-85e3-83bc0c0714b6 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:07.242632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345a6d5-ccc3-40bc-85e3-83bc0c0714b6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:07.242632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345a6d5-ccc3-40bc-85e3-83bc0c0714b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:07.267285 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8345a6d5-ccc3-40bc-85e3-83bc0c0714b6 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:07.268714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8345a6d5-ccc3-40bc-85e3-83bc0c0714b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:07.342677 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1278/2551] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:07.344222 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2617453-e3c9-4b4d-9371-9b85679728d5 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:07.344558 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2617453-e3c9-4b4d-9371-9b85679728d5 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:07.344859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2617453-e3c9-4b4d-9371-9b85679728d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:07.354477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:07.372098 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-e2617453-e3c9-4b4d-9371-9b85679728d5 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:07.373378 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e2617453-e3c9-4b4d-9371-9b85679728d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:07.383211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1274/2552] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:07.387906 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345a6d5-ccc3-40bc-85e3-83bc0c0714b6 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:07.400083 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345a6d5-ccc3-40bc-85e3-83bc0c0714b6 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:07.405879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901533c-0373-4cfb-b424-fa296254ff5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:07.405879 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901533c-0373-4cfb-b424-fa296254ff5c admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:07.406155 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901533c-0373-4cfb-b424-fa296254ff5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:07.406547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901533c-0373-4cfb-b424-fa296254ff5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:07.407307 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901533c-0373-4cfb-b424-fa296254ff5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:07.407668 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901533c-0373-4cfb-b424-fa296254ff5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:07.409443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901533c-0373-4cfb-b424-fa296254ff5c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:07.428937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1279/2553] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:07.430095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2617453-e3c9-4b4d-9371-9b85679728d5 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:07.440845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2617453-e3c9-4b4d-9371-9b85679728d5 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:07.447883 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180c477-9311-483e-b77d-985fff405025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:07.448575 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180c477-9311-483e-b77d-985fff405025 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:07.448854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180c477-9311-483e-b77d-985fff405025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:07.449123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180c477-9311-483e-b77d-985fff405025 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:07.449847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0180c477-9311-483e-b77d-985fff405025 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:07.451864 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0180c477-9311-483e-b77d-985fff405025 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:07.496037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1275/2554] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:07 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:56:07.498998 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901533c-0373-4cfb-b424-fa296254ff5c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:07.511514 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901533c-0373-4cfb-b424-fa296254ff5c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:07.594508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a0be1-2278-442c-8e48-60e0b4605318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:07.595142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a0be1-2278-442c-8e48-60e0b4605318 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:07.595474 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a0be1-2278-442c-8e48-60e0b4605318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:07.595738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a0be1-2278-442c-8e48-60e0b4605318 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:07.596389 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a0be1-2278-442c-8e48-60e0b4605318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:07.598482 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a0be1-2278-442c-8e48-60e0b4605318 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:07.607201 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:07.625026 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1280/2555] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:07 2022] POST /identity/v3/users => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:07.629784 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180c477-9311-483e-b77d-985fff405025 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:07.642690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180c477-9311-483e-b77d-985fff405025 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:07.648165 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c060a-46a2-4626-86f9-59b7c123e0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:07.648774 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c060a-46a2-4626-86f9-59b7c123e0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:07.649194 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c060a-46a2-4626-86f9-59b7c123e0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:07.649473 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c060a-46a2-4626-86f9-59b7c123e0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:07.650136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c060a-46a2-4626-86f9-59b7c123e0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:07.651751 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c060a-46a2-4626-86f9-59b7c123e0e7 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:07.665697 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1276/2556] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:07.669436 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a0be1-2278-442c-8e48-60e0b4605318 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:07.688323 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a0be1-2278-442c-8e48-60e0b4605318 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:07.695758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96a8e1-7695-4b56-b8a7-fd3d3b2fbd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:07.696413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96a8e1-7695-4b56-b8a7-fd3d3b2fbd4c admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:07.696702 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96a8e1-7695-4b56-b8a7-fd3d3b2fbd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:07.696998 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96a8e1-7695-4b56-b8a7-fd3d3b2fbd4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:07.697686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96a8e1-7695-4b56-b8a7-fd3d3b2fbd4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:07.699357 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96a8e1-7695-4b56-b8a7-fd3d3b2fbd4c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:07.712841 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1281/2557] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:07.718028 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c060a-46a2-4626-86f9-59b7c123e0e7 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:07.728878 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c060a-46a2-4626-86f9-59b7c123e0e7 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:07.734422 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5382be9-1273-4ac4-afdc-2ee3555d05b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:07.735150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5382be9-1273-4ac4-afdc-2ee3555d05b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:07.735436 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5382be9-1273-4ac4-afdc-2ee3555d05b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:07.735713 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5382be9-1273-4ac4-afdc-2ee3555d05b7 admin admin] PATH_INFO: `/v3/projects/f287e95dfa0544fb8fe25ea871c1f88c/users/b323b94dd2bf4ce384a296266b037c72/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:07.754597 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5382be9-1273-4ac4-afdc-2ee3555d05b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f287e95dfa0544fb8fe25ea871c1f88c, user_id=b323b94dd2bf4ce384a296266b037c72, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:07.757317 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5382be9-1273-4ac4-afdc-2ee3555d05b7 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:07.773165 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1277/2558] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:07 2022] PUT /identity/v3/projects/f287e95dfa0544fb8fe25ea871c1f88c/users/b323b94dd2bf4ce384a296266b037c72/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:56:07.783726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3525c61-1b31-47a9-9c93-2d34b499898b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:07.783936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3525c61-1b31-47a9-9c93-2d34b499898b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:07.784241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3525c61-1b31-47a9-9c93-2d34b499898b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:07.815365 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-b3525c61-1b31-47a9-9c93-2d34b499898b None None] MFA Rules not processed for user `b323b94dd2bf4ce384a296266b037c72`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:07.816622 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b3525c61-1b31-47a9-9c93-2d34b499898b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:07.883023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1282/2559] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:07.887802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5382be9-1273-4ac4-afdc-2ee3555d05b7 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:07.955319 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5382be9-1273-4ac4-afdc-2ee3555d05b7 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:07.959872 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d293f-88b5-48bc-8c59-b6b74eaaa46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:07.960508 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d293f-88b5-48bc-8c59-b6b74eaaa46f admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:07.960786 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d293f-88b5-48bc-8c59-b6b74eaaa46f admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:07.961056 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d293f-88b5-48bc-8c59-b6b74eaaa46f admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:07.961703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0d293f-88b5-48bc-8c59-b6b74eaaa46f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:07.963618 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0d293f-88b5-48bc-8c59-b6b74eaaa46f admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:07.994352 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1278/2560] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:07 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:56:07.995518 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3525c61-1b31-47a9-9c93-2d34b499898b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:08.007742 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3525c61-1b31-47a9-9c93-2d34b499898b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:08.068746 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9368f-8ab2-4ae3-8c62-6745051d4494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:08.069420 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9368f-8ab2-4ae3-8c62-6745051d4494 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:08.069696 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9368f-8ab2-4ae3-8c62-6745051d4494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:08.070020 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9368f-8ab2-4ae3-8c62-6745051d4494 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:08.070794 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9368f-8ab2-4ae3-8c62-6745051d4494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:08.073056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9368f-8ab2-4ae3-8c62-6745051d4494 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:08.096488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1283/2561] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:07 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:56:08.100944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d293f-88b5-48bc-8c59-b6b74eaaa46f admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:08.112681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d293f-88b5-48bc-8c59-b6b74eaaa46f admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:08.122150 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6179c0-4c83-4bc4-ade1-fdd7ff022979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:08.122779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6179c0-4c83-4bc4-ade1-fdd7ff022979 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:08.123050 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6179c0-4c83-4bc4-ade1-fdd7ff022979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:08.123314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6179c0-4c83-4bc4-ade1-fdd7ff022979 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:08.123968 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6179c0-4c83-4bc4-ade1-fdd7ff022979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:08.126359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6179c0-4c83-4bc4-ade1-fdd7ff022979 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:08.139977 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1279/2562] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:56:08.144237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9368f-8ab2-4ae3-8c62-6745051d4494 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:08.156171 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9368f-8ab2-4ae3-8c62-6745051d4494 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:08.164647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0597c5-1d31-465a-a1e1-c1ba10f92d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:08.165156 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0597c5-1d31-465a-a1e1-c1ba10f92d7e admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:08.165422 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0597c5-1d31-465a-a1e1-c1ba10f92d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:08.165691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0597c5-1d31-465a-a1e1-c1ba10f92d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:08.166395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0597c5-1d31-465a-a1e1-c1ba10f92d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:08.168038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0597c5-1d31-465a-a1e1-c1ba10f92d7e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:08.180036 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1284/2563] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:08.185322 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6179c0-4c83-4bc4-ade1-fdd7ff022979 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:08.197501 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6179c0-4c83-4bc4-ade1-fdd7ff022979 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:08.203412 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5a112-2ef8-431c-9205-2cfaf12dccb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:08.204291 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5a112-2ef8-431c-9205-2cfaf12dccb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:08.204572 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5a112-2ef8-431c-9205-2cfaf12dccb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:08.205122 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5a112-2ef8-431c-9205-2cfaf12dccb3 admin admin] PATH_INFO: `/v3/projects/85dad01c2314488aa9dc0fa07da25683/users/1bd538b88d184e47b948dc1bb5662a0e/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:08.236989 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a5a112-2ef8-431c-9205-2cfaf12dccb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85dad01c2314488aa9dc0fa07da25683, user_id=1bd538b88d184e47b948dc1bb5662a0e, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:08.243932 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a5a112-2ef8-431c-9205-2cfaf12dccb3 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:08.277228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1280/2564] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:08 2022] PUT /identity/v3/projects/85dad01c2314488aa9dc0fa07da25683/users/1bd538b88d184e47b948dc1bb5662a0e/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:08.278516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14b81d-4dcb-4d96-9235-abcd0bb8a628 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:08.279002 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14b81d-4dcb-4d96-9235-abcd0bb8a628 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:08.279120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14b81d-4dcb-4d96-9235-abcd0bb8a628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:08.324392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7e14b81d-4dcb-4d96-9235-abcd0bb8a628 None None] MFA Rules not processed for user `1bd538b88d184e47b948dc1bb5662a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:08.326310 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7e14b81d-4dcb-4d96-9235-abcd0bb8a628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:08.385902 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:08.431195 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1285/2565] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:56:08.436152 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5a112-2ef8-431c-9205-2cfaf12dccb3 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:08.489588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:08.511322 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5a112-2ef8-431c-9205-2cfaf12dccb3 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:08.517013 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1831c-56df-4105-a8b2-8ceb75a802a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:08.518012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1831c-56df-4105-a8b2-8ceb75a802a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:08.518463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1831c-56df-4105-a8b2-8ceb75a802a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:08.518736 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1831c-56df-4105-a8b2-8ceb75a802a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:08.519408 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac1831c-56df-4105-a8b2-8ceb75a802a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:08.522216 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac1831c-56df-4105-a8b2-8ceb75a802a1 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:08.556042 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1281/2566] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:08 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:08.558661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14b81d-4dcb-4d96-9235-abcd0bb8a628 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:08.570540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14b81d-4dcb-4d96-9235-abcd0bb8a628 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:08.649498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1713eef-a298-46bb-934b-07d0321c9826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:08.650214 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1713eef-a298-46bb-934b-07d0321c9826 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:08.650569 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1713eef-a298-46bb-934b-07d0321c9826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:08.650885 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1713eef-a298-46bb-934b-07d0321c9826 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:08.651585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1713eef-a298-46bb-934b-07d0321c9826 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:08.653615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1713eef-a298-46bb-934b-07d0321c9826 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:08.680130 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1286/2567] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:08 2022] POST /identity/v3/users => generated 296 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:08.685036 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1831c-56df-4105-a8b2-8ceb75a802a1 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:08.702078 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1831c-56df-4105-a8b2-8ceb75a802a1 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:08.709633 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d932b7f-de35-4ef9-9953-e2bad21f0dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:08.710275 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d932b7f-de35-4ef9-9953-e2bad21f0dea admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:08.710550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d932b7f-de35-4ef9-9953-e2bad21f0dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:08.710839 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d932b7f-de35-4ef9-9953-e2bad21f0dea admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:08.711498 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d932b7f-de35-4ef9-9953-e2bad21f0dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:08.713109 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d932b7f-de35-4ef9-9953-e2bad21f0dea admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:08.726312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1282/2568] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:08.731005 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1713eef-a298-46bb-934b-07d0321c9826 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:08.742934 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1713eef-a298-46bb-934b-07d0321c9826 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:08.747865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe2720-fa40-41b1-ad18-3f595f498134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:08.748607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe2720-fa40-41b1-ad18-3f595f498134 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:08.748941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe2720-fa40-41b1-ad18-3f595f498134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:08.749179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe2720-fa40-41b1-ad18-3f595f498134 admin admin] PATH_INFO: `/v3/projects/37c1b676e79b4b96a7d3453d589e19ee/users/ce47cf5e763849cfbcd99351c258ee0f/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:08.770038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe2720-fa40-41b1-ad18-3f595f498134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37c1b676e79b4b96a7d3453d589e19ee, user_id=ce47cf5e763849cfbcd99351c258ee0f, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:08.773141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe2720-fa40-41b1-ad18-3f595f498134 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:08.788473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1287/2569] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:08 2022] PUT /identity/v3/projects/37c1b676e79b4b96a7d3453d589e19ee/users/ce47cf5e763849cfbcd99351c258ee0f/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:08.792997 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d932b7f-de35-4ef9-9953-e2bad21f0dea admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:08.804913 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d932b7f-de35-4ef9-9953-e2bad21f0dea admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:08.878736 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eeec71-4a98-4a8c-9c30-f6b7bd69aca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:08.879667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eeec71-4a98-4a8c-9c30-f6b7bd69aca5 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:08.880191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eeec71-4a98-4a8c-9c30-f6b7bd69aca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:08.880548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eeec71-4a98-4a8c-9c30-f6b7bd69aca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:08.882423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eeec71-4a98-4a8c-9c30-f6b7bd69aca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:08.883128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eeec71-4a98-4a8c-9c30-f6b7bd69aca5 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:08.899267 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1283/2570] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:08.903509 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe2720-fa40-41b1-ad18-3f595f498134 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:08.915080 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe2720-fa40-41b1-ad18-3f595f498134 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:08.924228 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748aa52-e30d-477d-9c86-0eed4f2a1cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:08.925622 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748aa52-e30d-477d-9c86-0eed4f2a1cd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:08.926021 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748aa52-e30d-477d-9c86-0eed4f2a1cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:08.926420 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748aa52-e30d-477d-9c86-0eed4f2a1cd8 admin admin] PATH_INFO: `/v3/projects/37c1b676e79b4b96a7d3453d589e19ee/users/ce47cf5e763849cfbcd99351c258ee0f/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:08.929995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748aa52-e30d-477d-9c86-0eed4f2a1cd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37c1b676e79b4b96a7d3453d589e19ee, user_id=ce47cf5e763849cfbcd99351c258ee0f, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:08.932763 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748aa52-e30d-477d-9c86-0eed4f2a1cd8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:08.954322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1288/2571] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:08 2022] PUT /identity/v3/projects/37c1b676e79b4b96a7d3453d589e19ee/users/ce47cf5e763849cfbcd99351c258ee0f/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:08.957717 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e6a35-2d7b-4d37-ad5b-d0c8827e5305 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:08.958000 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e6a35-2d7b-4d37-ad5b-d0c8827e5305 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:08.958269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e6a35-2d7b-4d37-ad5b-d0c8827e5305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:08.989683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-966e6a35-2d7b-4d37-ad5b-d0c8827e5305 None None] MFA Rules not processed for user `ce47cf5e763849cfbcd99351c258ee0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:08.990982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-966e6a35-2d7b-4d37-ad5b-d0c8827e5305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:09.069752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1284/2572] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:09.082020 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748aa52-e30d-477d-9c86-0eed4f2a1cd8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:09.083846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9748aa52-e30d-477d-9c86-0eed4f2a1cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:09.170265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748aa52-e30d-477d-9c86-0eed4f2a1cd8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:09.182207 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb9b27-2397-4d34-a20e-16fd411ec2b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:09.183513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb9b27-2397-4d34-a20e-16fd411ec2b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:09.184087 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb9b27-2397-4d34-a20e-16fd411ec2b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:09.184638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb9b27-2397-4d34-a20e-16fd411ec2b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:09.205715 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb9b27-2397-4d34-a20e-16fd411ec2b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:09.208586 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb9b27-2397-4d34-a20e-16fd411ec2b1 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:09.228412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1289/2573] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:09 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:09.286718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e6a35-2d7b-4d37-ad5b-d0c8827e5305 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:09.288391 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-966e6a35-2d7b-4d37-ad5b-d0c8827e5305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:09.301920 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e6a35-2d7b-4d37-ad5b-d0c8827e5305 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:09.305744 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5130a62-5a42-4f83-b258-39ccce96c334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:09.306366 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5130a62-5a42-4f83-b258-39ccce96c334 service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:09.306652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5130a62-5a42-4f83-b258-39ccce96c334 service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:09.306913 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5130a62-5a42-4f83-b258-39ccce96c334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:09.362498 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:09.384899 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5130a62-5a42-4f83-b258-39ccce96c334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:09.386861 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5130a62-5a42-4f83-b258-39ccce96c334 service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:09.400857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1285/2574] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:12.416149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb9b27-2397-4d34-a20e-16fd411ec2b1 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:12.451732 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:12.484382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb9b27-2397-4d34-a20e-16fd411ec2b1 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:12.489975 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9ba47-f5a7-42e9-88d6-17d5e8fbea84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:12.490599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9ba47-f5a7-42e9-88d6-17d5e8fbea84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:12.490870 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9ba47-f5a7-42e9-88d6-17d5e8fbea84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:12.491131 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9ba47-f5a7-42e9-88d6-17d5e8fbea84 admin admin] PATH_INFO: `/v3/users/b323b94dd2bf4ce384a296266b037c72` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:12.492347 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee9ba47-f5a7-42e9-88d6-17d5e8fbea84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b323b94dd2bf4ce384a296266b037c72)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:12.494267 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee9ba47-f5a7-42e9-88d6-17d5e8fbea84 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:12.546946 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-cee9ba47-f5a7-42e9-88d6-17d5e8fbea84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b323b94dd2bf4ce384a296266b037c72'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:12.554839 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-cee9ba47-f5a7-42e9-88d6-17d5e8fbea84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b323b94dd2bf4ce384a296266b037c72'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:12.563649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-cee9ba47-f5a7-42e9-88d6-17d5e8fbea84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b323b94dd2bf4ce384a296266b037c72'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:12.564274 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-cee9ba47-f5a7-42e9-88d6-17d5e8fbea84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b323b94dd2bf4ce384a296266b037c72'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:12.577086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-cee9ba47-f5a7-42e9-88d6-17d5e8fbea84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b323b94dd2bf4ce384a296266b037c72'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:12.587422 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1290/2575] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:12 2022] DELETE /identity/v3/users/b323b94dd2bf4ce384a296266b037c72 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:12.592308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca5e2b-2122-4575-90c9-d49d13c7c4cc None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:12.592792 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca5e2b-2122-4575-90c9-d49d13c7c4cc None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:12.592890 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca5e2b-2122-4575-90c9-d49d13c7c4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:12.612770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-90ca5e2b-2122-4575-90c9-d49d13c7c4cc None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:12.614088 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-90ca5e2b-2122-4575-90c9-d49d13c7c4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:12.683098 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1286/2576] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:12.696159 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9ba47-f5a7-42e9-88d6-17d5e8fbea84 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:12.698096 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cee9ba47-f5a7-42e9-88d6-17d5e8fbea84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:12.762375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9ba47-f5a7-42e9-88d6-17d5e8fbea84 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:12.765940 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56389f09-2755-4e19-be98-4a97c5265186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:12.766560 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56389f09-2755-4e19-be98-4a97c5265186 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:12.766834 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56389f09-2755-4e19-be98-4a97c5265186 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:12.767109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56389f09-2755-4e19-be98-4a97c5265186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:12.777163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56389f09-2755-4e19-be98-4a97c5265186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:12.778622 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56389f09-2755-4e19-be98-4a97c5265186 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:12.790391 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1291/2577] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:13.089229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca5e2b-2122-4575-90c9-d49d13c7c4cc None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:13.090679 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-90ca5e2b-2122-4575-90c9-d49d13c7c4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:13.112305 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca5e2b-2122-4575-90c9-d49d13c7c4cc None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:13.127029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:13.127694 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:13.127973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:13.128276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e admin admin] PATH_INFO: `/v3/projects/f287e95dfa0544fb8fe25ea871c1f88c` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:13.129449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f287e95dfa0544fb8fe25ea871c1f88c)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:13.132046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:13.183128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f287e95dfa0544fb8fe25ea871c1f88c', 'request_id': 'req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:13.189011 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f287e95dfa0544fb8fe25ea871c1f88c', 'request_id': 'req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:13.195888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f287e95dfa0544fb8fe25ea871c1f88c', 'request_id': 'req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:13.211160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e admin admin] The token cache is being invalidate because project f287e95dfa0544fb8fe25ea871c1f88c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:13.211582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:13.212260 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:13.213846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1287/2578] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:13 2022] DELETE /identity/v3/projects/f287e95dfa0544fb8fe25ea871c1f88c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:13.217897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56389f09-2755-4e19-be98-4a97c5265186 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:13.227864 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56389f09-2755-4e19-be98-4a97c5265186 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:13.291225 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:13.301931 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de8f9c-79c4-4832-bbb5-6c9a65627cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:13.302739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de8f9c-79c4-4832-bbb5-6c9a65627cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:13.303114 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de8f9c-79c4-4832-bbb5-6c9a65627cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:13.303557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de8f9c-79c4-4832-bbb5-6c9a65627cf8 admin admin] PATH_INFO: `/v3/users/1bd538b88d184e47b948dc1bb5662a0e` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:13.305029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de8f9c-79c4-4832-bbb5-6c9a65627cf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bd538b88d184e47b948dc1bb5662a0e)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:13.307003 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de8f9c-79c4-4832-bbb5-6c9a65627cf8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:13.364377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e9de8f9c-79c4-4832-bbb5-6c9a65627cf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bd538b88d184e47b948dc1bb5662a0e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:13.372334 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e9de8f9c-79c4-4832-bbb5-6c9a65627cf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bd538b88d184e47b948dc1bb5662a0e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:13.383082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e9de8f9c-79c4-4832-bbb5-6c9a65627cf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bd538b88d184e47b948dc1bb5662a0e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:13.383692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e9de8f9c-79c4-4832-bbb5-6c9a65627cf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bd538b88d184e47b948dc1bb5662a0e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:13.396132 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e9de8f9c-79c4-4832-bbb5-6c9a65627cf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bd538b88d184e47b948dc1bb5662a0e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:13.402093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1292/2579] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:13 2022] DELETE /identity/v3/users/1bd538b88d184e47b948dc1bb5662a0e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:13.712455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:13.713956 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:13.728367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9fb0e-1fdb-4f46-912d-6baa4e1dee1e admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:13.788871 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d510235-b0ee-4ced-aab3-61fee69c307d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:13.789483 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d510235-b0ee-4ced-aab3-61fee69c307d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:13.789754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d510235-b0ee-4ced-aab3-61fee69c307d admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:13.790020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d510235-b0ee-4ced-aab3-61fee69c307d admin admin] PATH_INFO: `/v3/projects/85dad01c2314488aa9dc0fa07da25683` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:13.791230 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d510235-b0ee-4ced-aab3-61fee69c307d admin admin] RBAC: Authorizing `identity:delete_project(project_id=85dad01c2314488aa9dc0fa07da25683)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:13.793194 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d510235-b0ee-4ced-aab3-61fee69c307d admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:13.801871 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:13.849193 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-9d510235-b0ee-4ced-aab3-61fee69c307d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85dad01c2314488aa9dc0fa07da25683', 'request_id': 'req-9d510235-b0ee-4ced-aab3-61fee69c307d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:13.855787 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-9d510235-b0ee-4ced-aab3-61fee69c307d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85dad01c2314488aa9dc0fa07da25683', 'request_id': 'req-9d510235-b0ee-4ced-aab3-61fee69c307d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:13.864671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-9d510235-b0ee-4ced-aab3-61fee69c307d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85dad01c2314488aa9dc0fa07da25683', 'request_id': 'req-9d510235-b0ee-4ced-aab3-61fee69c307d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:13.879708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-9d510235-b0ee-4ced-aab3-61fee69c307d admin admin] The token cache is being invalidate because project 85dad01c2314488aa9dc0fa07da25683 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:13.880132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-9d510235-b0ee-4ced-aab3-61fee69c307d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:13.880796 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-9d510235-b0ee-4ced-aab3-61fee69c307d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:13.882335 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1288/2580] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:13 2022] DELETE /identity/v3/projects/85dad01c2314488aa9dc0fa07da25683 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:13.887056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de8f9c-79c4-4832-bbb5-6c9a65627cf8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:13.890089 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e9de8f9c-79c4-4832-bbb5-6c9a65627cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:13.904158 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de8f9c-79c4-4832-bbb5-6c9a65627cf8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:13.966339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298ed3c-452b-4043-b4e1-6ae98050b8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:13.966886 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298ed3c-452b-4043-b4e1-6ae98050b8d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:13.967185 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298ed3c-452b-4043-b4e1-6ae98050b8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:13.967452 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298ed3c-452b-4043-b4e1-6ae98050b8d3 admin admin] PATH_INFO: `/v3/users/ce47cf5e763849cfbcd99351c258ee0f` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:13.968675 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298ed3c-452b-4043-b4e1-6ae98050b8d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce47cf5e763849cfbcd99351c258ee0f)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:13.970607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298ed3c-452b-4043-b4e1-6ae98050b8d3 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:14.020128 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2298ed3c-452b-4043-b4e1-6ae98050b8d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce47cf5e763849cfbcd99351c258ee0f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:14.027067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2298ed3c-452b-4043-b4e1-6ae98050b8d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce47cf5e763849cfbcd99351c258ee0f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:14.035129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2298ed3c-452b-4043-b4e1-6ae98050b8d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce47cf5e763849cfbcd99351c258ee0f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:14.035913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2298ed3c-452b-4043-b4e1-6ae98050b8d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce47cf5e763849cfbcd99351c258ee0f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:14.044406 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:14.049804 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2298ed3c-452b-4043-b4e1-6ae98050b8d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce47cf5e763849cfbcd99351c258ee0f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:14.057987 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1293/2581] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:13 2022] DELETE /identity/v3/users/ce47cf5e763849cfbcd99351c258ee0f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:14.518760 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d510235-b0ee-4ced-aab3-61fee69c307d admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:14.521067 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-9d510235-b0ee-4ced-aab3-61fee69c307d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:14.546317 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d510235-b0ee-4ced-aab3-61fee69c307d admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:14.619847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce3ee2-b4cd-497a-bf1b-9ed3fa014ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:14.620514 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce3ee2-b4cd-497a-bf1b-9ed3fa014ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:14.620790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce3ee2-b4cd-497a-bf1b-9ed3fa014ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:14.621055 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce3ee2-b4cd-497a-bf1b-9ed3fa014ad4 admin admin] PATH_INFO: `/v3/projects/37c1b676e79b4b96a7d3453d589e19ee` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:14.625992 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce3ee2-b4cd-497a-bf1b-9ed3fa014ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37c1b676e79b4b96a7d3453d589e19ee)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:14.627979 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce3ee2-b4cd-497a-bf1b-9ed3fa014ad4 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:14.686803 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-25ce3ee2-b4cd-497a-bf1b-9ed3fa014ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37c1b676e79b4b96a7d3453d589e19ee', 'request_id': 'req-25ce3ee2-b4cd-497a-bf1b-9ed3fa014ad4'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:14.694979 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-25ce3ee2-b4cd-497a-bf1b-9ed3fa014ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37c1b676e79b4b96a7d3453d589e19ee', 'request_id': 'req-25ce3ee2-b4cd-497a-bf1b-9ed3fa014ad4'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:14.702917 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-25ce3ee2-b4cd-497a-bf1b-9ed3fa014ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37c1b676e79b4b96a7d3453d589e19ee', 'request_id': 'req-25ce3ee2-b4cd-497a-bf1b-9ed3fa014ad4'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:14.717800 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-25ce3ee2-b4cd-497a-bf1b-9ed3fa014ad4 admin admin] The token cache is being invalidate because project 37c1b676e79b4b96a7d3453d589e19ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:14.718502 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-25ce3ee2-b4cd-497a-bf1b-9ed3fa014ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:14.718708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-25ce3ee2-b4cd-497a-bf1b-9ed3fa014ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:14.720560 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1289/2582] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:14 2022] DELETE /identity/v3/projects/37c1b676e79b4b96a7d3453d589e19ee => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:14.726569 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f394f3d-adf8-4a0e-9736-b5760124924f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:14.726753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f394f3d-adf8-4a0e-9736-b5760124924f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:14.727028 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f394f3d-adf8-4a0e-9736-b5760124924f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:14.759525 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-6f394f3d-adf8-4a0e-9736-b5760124924f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:14.760710 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6f394f3d-adf8-4a0e-9736-b5760124924f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:14.838345 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1294/2583] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:56:14.848453 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0d10a-e011-4231-873d-1f33c44ee161 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:14.848786 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0d10a-e011-4231-873d-1f33c44ee161 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:14.849012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0d10a-e011-4231-873d-1f33c44ee161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:14.872456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-3df0d10a-e011-4231-873d-1f33c44ee161 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:14.873707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-3df0d10a-e011-4231-873d-1f33c44ee161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:14.882793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1290/2584] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:14.887280 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f394f3d-adf8-4a0e-9736-b5760124924f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:14.897533 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f394f3d-adf8-4a0e-9736-b5760124924f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:14.902709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cb681-153b-47e4-b732-51c58597db59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:14.903614 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cb681-153b-47e4-b732-51c58597db59 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:14.903889 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cb681-153b-47e4-b732-51c58597db59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:14.904203 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cb681-153b-47e4-b732-51c58597db59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:14.904862 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26cb681-153b-47e4-b732-51c58597db59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:14.905167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26cb681-153b-47e4-b732-51c58597db59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:14.906768 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26cb681-153b-47e4-b732-51c58597db59 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:14.932149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1295/2585] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:56:14.936802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0d10a-e011-4231-873d-1f33c44ee161 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:14.950020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0d10a-e011-4231-873d-1f33c44ee161 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:14.955840 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f700fdb-f810-402e-98d3-99bee637fe3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:14.956690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f700fdb-f810-402e-98d3-99bee637fe3a admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:14.957094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f700fdb-f810-402e-98d3-99bee637fe3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:14.958389 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f700fdb-f810-402e-98d3-99bee637fe3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:14.959193 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f700fdb-f810-402e-98d3-99bee637fe3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:14.962358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f700fdb-f810-402e-98d3-99bee637fe3a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:14.992860 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:15.007763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1291/2586] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:14 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:15.013251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cb681-153b-47e4-b732-51c58597db59 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:15.025861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cb681-153b-47e4-b732-51c58597db59 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:15.092637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02147a7-8f61-4985-87ff-4d011060595c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:15.093259 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02147a7-8f61-4985-87ff-4d011060595c admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:15.093531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02147a7-8f61-4985-87ff-4d011060595c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:15.093940 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02147a7-8f61-4985-87ff-4d011060595c admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:15.094712 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02147a7-8f61-4985-87ff-4d011060595c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:15.097449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02147a7-8f61-4985-87ff-4d011060595c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:15.126429 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1296/2587] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:15 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:15.128147 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f700fdb-f810-402e-98d3-99bee637fe3a admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:15.139931 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f700fdb-f810-402e-98d3-99bee637fe3a admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:15.144978 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c370e94-5b08-4539-a99d-e4d5f23a47ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:15.145661 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c370e94-5b08-4539-a99d-e4d5f23a47ea admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:15.145967 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c370e94-5b08-4539-a99d-e4d5f23a47ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:15.146228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c370e94-5b08-4539-a99d-e4d5f23a47ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:15.147015 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c370e94-5b08-4539-a99d-e4d5f23a47ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:15.149432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c370e94-5b08-4539-a99d-e4d5f23a47ea admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:15.167693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1292/2588] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:15.169236 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02147a7-8f61-4985-87ff-4d011060595c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:15.180268 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02147a7-8f61-4985-87ff-4d011060595c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:15.184858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3b8fe-48c2-43bf-9aaf-3db173ab3dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:15.185469 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3b8fe-48c2-43bf-9aaf-3db173ab3dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:15.185741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3b8fe-48c2-43bf-9aaf-3db173ab3dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:15.186103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3b8fe-48c2-43bf-9aaf-3db173ab3dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:15.186743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3b8fe-48c2-43bf-9aaf-3db173ab3dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:15.188392 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3b8fe-48c2-43bf-9aaf-3db173ab3dc1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:15.200741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1297/2589] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:15.205668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c370e94-5b08-4539-a99d-e4d5f23a47ea admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:15.216940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c370e94-5b08-4539-a99d-e4d5f23a47ea admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:15.225382 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49caac-566a-449d-99dd-4e2f7047280f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:15.226069 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49caac-566a-449d-99dd-4e2f7047280f admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:15.226340 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49caac-566a-449d-99dd-4e2f7047280f admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:15.226723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49caac-566a-449d-99dd-4e2f7047280f admin admin] PATH_INFO: `/v3/projects/c70c92977d2e401893930f31549be260/users/03b99fa39ee74209aedf20407b37c030/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:15.244862 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f49caac-566a-449d-99dd-4e2f7047280f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c70c92977d2e401893930f31549be260, user_id=03b99fa39ee74209aedf20407b37c030, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:15.248698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f49caac-566a-449d-99dd-4e2f7047280f admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:15.264387 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1293/2590] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:15 2022] PUT /identity/v3/projects/c70c92977d2e401893930f31549be260/users/03b99fa39ee74209aedf20407b37c030/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:15.276588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d4638-20b3-43db-9b4d-1aca293745cd None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:15.276867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d4638-20b3-43db-9b4d-1aca293745cd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:15.277135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d4638-20b3-43db-9b4d-1aca293745cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:15.305602 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-444d4638-20b3-43db-9b4d-1aca293745cd None None] MFA Rules not processed for user `03b99fa39ee74209aedf20407b37c030`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:15.306911 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-444d4638-20b3-43db-9b4d-1aca293745cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:15.365657 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:15.394508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1298/2591] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:15.399139 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49caac-566a-449d-99dd-4e2f7047280f admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:15.503538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49caac-566a-449d-99dd-4e2f7047280f admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:15.511015 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8149a-304d-4da2-ba79-c89f5db445af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:15.512120 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8149a-304d-4da2-ba79-c89f5db445af admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:15.512470 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8149a-304d-4da2-ba79-c89f5db445af admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:15.513274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8149a-304d-4da2-ba79-c89f5db445af admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:15.514086 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8149a-304d-4da2-ba79-c89f5db445af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:15.518431 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8149a-304d-4da2-ba79-c89f5db445af admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:15.567420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1294/2592] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:15 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:15.572663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d4638-20b3-43db-9b4d-1aca293745cd None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:15.585858 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d4638-20b3-43db-9b4d-1aca293745cd None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:15.678255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2268c-f636-4f0e-9073-e054e606cd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:15.678902 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2268c-f636-4f0e-9073-e054e606cd48 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:15.679179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2268c-f636-4f0e-9073-e054e606cd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:15.679441 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2268c-f636-4f0e-9073-e054e606cd48 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:15.680106 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2268c-f636-4f0e-9073-e054e606cd48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:15.683499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2268c-f636-4f0e-9073-e054e606cd48 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:15.714757 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1299/2593] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:15 2022] POST /identity/v3/users => generated 298 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:15.718781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8149a-304d-4da2-ba79-c89f5db445af admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:15.731095 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8149a-304d-4da2-ba79-c89f5db445af admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:15.736247 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12e528-4cc0-45fd-bcfd-cb0cf5c4220b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:15.736975 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12e528-4cc0-45fd-bcfd-cb0cf5c4220b admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:15.737323 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12e528-4cc0-45fd-bcfd-cb0cf5c4220b admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:15.737665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12e528-4cc0-45fd-bcfd-cb0cf5c4220b admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:15.738410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee12e528-4cc0-45fd-bcfd-cb0cf5c4220b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:15.740535 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee12e528-4cc0-45fd-bcfd-cb0cf5c4220b admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:15.758475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1295/2594] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:56:15.759901 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2268c-f636-4f0e-9073-e054e606cd48 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:15.771609 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2268c-f636-4f0e-9073-e054e606cd48 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:15.776663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89116e-a15e-41e0-8a8f-85d9f2e3393b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:15.777186 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89116e-a15e-41e0-8a8f-85d9f2e3393b admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:15.777470 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89116e-a15e-41e0-8a8f-85d9f2e3393b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:15.777748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89116e-a15e-41e0-8a8f-85d9f2e3393b admin admin] PATH_INFO: `/v3/projects/c6dc5d9dd1d24232a3e766417e80a3a1/users/5d3a258f3eaa4a539951317c141c005d/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:15.805122 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d89116e-a15e-41e0-8a8f-85d9f2e3393b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6dc5d9dd1d24232a3e766417e80a3a1, user_id=5d3a258f3eaa4a539951317c141c005d, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:15.807865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d89116e-a15e-41e0-8a8f-85d9f2e3393b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:15.822562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1300/2595] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:15 2022] PUT /identity/v3/projects/c6dc5d9dd1d24232a3e766417e80a3a1/users/5d3a258f3eaa4a539951317c141c005d/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:15.826923 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12e528-4cc0-45fd-bcfd-cb0cf5c4220b admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:15.838887 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12e528-4cc0-45fd-bcfd-cb0cf5c4220b admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:15.906406 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5f534-1704-49d0-9f9a-e1f73f369a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:15.907024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5f534-1704-49d0-9f9a-e1f73f369a86 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:15.907294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5f534-1704-49d0-9f9a-e1f73f369a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:15.907564 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5f534-1704-49d0-9f9a-e1f73f369a86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:15.908241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5f534-1704-49d0-9f9a-e1f73f369a86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:15.909831 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5f534-1704-49d0-9f9a-e1f73f369a86 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:15.922075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1296/2596] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:15.926915 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89116e-a15e-41e0-8a8f-85d9f2e3393b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:15.939322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89116e-a15e-41e0-8a8f-85d9f2e3393b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:15.945498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf63e5-3796-4074-a776-c458c4257a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:15.946175 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf63e5-3796-4074-a776-c458c4257a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:15.946449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf63e5-3796-4074-a776-c458c4257a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:15.946713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf63e5-3796-4074-a776-c458c4257a41 admin admin] PATH_INFO: `/v3/projects/c6dc5d9dd1d24232a3e766417e80a3a1/users/5d3a258f3eaa4a539951317c141c005d/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:15.948703 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf63e5-3796-4074-a776-c458c4257a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6dc5d9dd1d24232a3e766417e80a3a1, user_id=5d3a258f3eaa4a539951317c141c005d, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:15.951634 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf63e5-3796-4074-a776-c458c4257a41 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:15.963688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1301/2597] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:15 2022] PUT /identity/v3/projects/c6dc5d9dd1d24232a3e766417e80a3a1/users/5d3a258f3eaa4a539951317c141c005d/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:16.035735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34443aa5-c0a0-423c-9de0-0b1df05a5bde None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:16.036063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34443aa5-c0a0-423c-9de0-0b1df05a5bde None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:16.036223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34443aa5-c0a0-423c-9de0-0b1df05a5bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:16.071031 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-34443aa5-c0a0-423c-9de0-0b1df05a5bde None None] MFA Rules not processed for user `5d3a258f3eaa4a539951317c141c005d`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:16.072311 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-34443aa5-c0a0-423c-9de0-0b1df05a5bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:16.099770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:16.170806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1297/2598] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:16.188621 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf63e5-3796-4074-a776-c458c4257a41 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:16.191439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f6cf63e5-3796-4074-a776-c458c4257a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:16.264076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf63e5-3796-4074-a776-c458c4257a41 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:16.273261 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c800b69-f821-4f0d-aa08-0da0687860b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:16.274039 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c800b69-f821-4f0d-aa08-0da0687860b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:16.274490 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c800b69-f821-4f0d-aa08-0da0687860b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:16.274966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c800b69-f821-4f0d-aa08-0da0687860b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:16.285339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c800b69-f821-4f0d-aa08-0da0687860b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:16.287403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c800b69-f821-4f0d-aa08-0da0687860b5 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:16.305269 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1302/2599] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:16 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:16.424538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34443aa5-c0a0-423c-9de0-0b1df05a5bde None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:16.436099 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34443aa5-c0a0-423c-9de0-0b1df05a5bde None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:16.441933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84289632-c41a-4e4b-b543-bc1ce499f434 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:16.442895 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84289632-c41a-4e4b-b543-bc1ce499f434 service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:16.443932 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84289632-c41a-4e4b-b543-bc1ce499f434 service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:16.443932 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84289632-c41a-4e4b-b543-bc1ce499f434 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:16.511131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84289632-c41a-4e4b-b543-bc1ce499f434 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:16.512626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84289632-c41a-4e4b-b543-bc1ce499f434 service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:16.522140 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1298/2600] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:16.831291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c800b69-f821-4f0d-aa08-0da0687860b5 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:16.882914 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:16.903096 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c800b69-f821-4f0d-aa08-0da0687860b5 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:16.908793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a6ad6-ab27-48ee-9e72-155e5fa532bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:16.915574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a6ad6-ab27-48ee-9e72-155e5fa532bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:16.916106 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a6ad6-ab27-48ee-9e72-155e5fa532bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:16.916511 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a6ad6-ab27-48ee-9e72-155e5fa532bc admin admin] PATH_INFO: `/v3/users/03b99fa39ee74209aedf20407b37c030` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:16.917932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7a6ad6-ab27-48ee-9e72-155e5fa532bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=03b99fa39ee74209aedf20407b37c030)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:16.919991 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7a6ad6-ab27-48ee-9e72-155e5fa532bc admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:16.989829 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8f7a6ad6-ab27-48ee-9e72-155e5fa532bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03b99fa39ee74209aedf20407b37c030'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:16.997167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8f7a6ad6-ab27-48ee-9e72-155e5fa532bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03b99fa39ee74209aedf20407b37c030'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:17.006014 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8f7a6ad6-ab27-48ee-9e72-155e5fa532bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03b99fa39ee74209aedf20407b37c030'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:17.006640 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8f7a6ad6-ab27-48ee-9e72-155e5fa532bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03b99fa39ee74209aedf20407b37c030'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:17.019050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8f7a6ad6-ab27-48ee-9e72-155e5fa532bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03b99fa39ee74209aedf20407b37c030'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:17.026373 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1303/2601] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:16 2022] DELETE /identity/v3/users/03b99fa39ee74209aedf20407b37c030 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:17.030903 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82581e0a-5497-48d2-8ada-385f701f55be None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:17.031403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82581e0a-5497-48d2-8ada-385f701f55be None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:17.031504 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82581e0a-5497-48d2-8ada-385f701f55be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:17.052749 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-82581e0a-5497-48d2-8ada-385f701f55be None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:17.053898 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-82581e0a-5497-48d2-8ada-385f701f55be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:17.122069 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1299/2602] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:17.132104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a6ad6-ab27-48ee-9e72-155e5fa532bc admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:17.133586 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8f7a6ad6-ab27-48ee-9e72-155e5fa532bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:17.192796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a6ad6-ab27-48ee-9e72-155e5fa532bc admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:17.196798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605ce10-a45c-4adc-8587-09aa63bc19dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:17.197617 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605ce10-a45c-4adc-8587-09aa63bc19dd service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:17.198009 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605ce10-a45c-4adc-8587-09aa63bc19dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:17.198283 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605ce10-a45c-4adc-8587-09aa63bc19dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:17.208015 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605ce10-a45c-4adc-8587-09aa63bc19dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:17.209484 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605ce10-a45c-4adc-8587-09aa63bc19dd service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:17.220974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1304/2603] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:17.548628 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82581e0a-5497-48d2-8ada-385f701f55be None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:17.550654 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-82581e0a-5497-48d2-8ada-385f701f55be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:17.563758 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82581e0a-5497-48d2-8ada-385f701f55be None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:17.569264 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6a141-5885-4361-a389-966302bf78b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:17.570037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6a141-5885-4361-a389-966302bf78b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:17.570444 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6a141-5885-4361-a389-966302bf78b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:17.571297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6a141-5885-4361-a389-966302bf78b4 admin admin] PATH_INFO: `/v3/projects/c70c92977d2e401893930f31549be260` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:17.572596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6a141-5885-4361-a389-966302bf78b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c70c92977d2e401893930f31549be260)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:17.574650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6a141-5885-4361-a389-966302bf78b4 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:17.628821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-27b6a141-5885-4361-a389-966302bf78b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c70c92977d2e401893930f31549be260', 'request_id': 'req-27b6a141-5885-4361-a389-966302bf78b4'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:17.631423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:17.636589 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-27b6a141-5885-4361-a389-966302bf78b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c70c92977d2e401893930f31549be260', 'request_id': 'req-27b6a141-5885-4361-a389-966302bf78b4'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:17.643567 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-27b6a141-5885-4361-a389-966302bf78b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c70c92977d2e401893930f31549be260', 'request_id': 'req-27b6a141-5885-4361-a389-966302bf78b4'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:17.657695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-27b6a141-5885-4361-a389-966302bf78b4 admin admin] The token cache is being invalidate because project c70c92977d2e401893930f31549be260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:17.657964 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-27b6a141-5885-4361-a389-966302bf78b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:17.658605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-27b6a141-5885-4361-a389-966302bf78b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:17.660164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1300/2604] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:17 2022] DELETE /identity/v3/projects/c70c92977d2e401893930f31549be260 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:17.664755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605ce10-a45c-4adc-8587-09aa63bc19dd service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:17.675474 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605ce10-a45c-4adc-8587-09aa63bc19dd service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:17.735541 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d65ba4-bdd7-4b4b-bbde-bee96539a9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:17.736199 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d65ba4-bdd7-4b4b-bbde-bee96539a9be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:17.736480 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d65ba4-bdd7-4b4b-bbde-bee96539a9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:17.736760 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d65ba4-bdd7-4b4b-bbde-bee96539a9be admin admin] PATH_INFO: `/v3/users/5d3a258f3eaa4a539951317c141c005d` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:17.737875 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d65ba4-bdd7-4b4b-bbde-bee96539a9be admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d3a258f3eaa4a539951317c141c005d)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:17.740451 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d65ba4-bdd7-4b4b-bbde-bee96539a9be admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:17.745455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:17.799711 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-20d65ba4-bdd7-4b4b-bbde-bee96539a9be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d3a258f3eaa4a539951317c141c005d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:17.808012 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-20d65ba4-bdd7-4b4b-bbde-bee96539a9be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d3a258f3eaa4a539951317c141c005d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:17.817605 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-20d65ba4-bdd7-4b4b-bbde-bee96539a9be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d3a258f3eaa4a539951317c141c005d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:17.818145 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-20d65ba4-bdd7-4b4b-bbde-bee96539a9be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d3a258f3eaa4a539951317c141c005d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:17.832375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-20d65ba4-bdd7-4b4b-bbde-bee96539a9be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d3a258f3eaa4a539951317c141c005d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:17.838574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1305/2605] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:17 2022] DELETE /identity/v3/users/5d3a258f3eaa4a539951317c141c005d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:18.125913 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6a141-5885-4361-a389-966302bf78b4 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:18.127533 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-27b6a141-5885-4361-a389-966302bf78b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:18.143659 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6a141-5885-4361-a389-966302bf78b4 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:18.216861 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39f36d-7886-47b1-bb6f-9de334981909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:18.217529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39f36d-7886-47b1-bb6f-9de334981909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:18.217796 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39f36d-7886-47b1-bb6f-9de334981909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:18.218063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39f36d-7886-47b1-bb6f-9de334981909 admin admin] PATH_INFO: `/v3/projects/c6dc5d9dd1d24232a3e766417e80a3a1` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:18.219219 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39f36d-7886-47b1-bb6f-9de334981909 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6dc5d9dd1d24232a3e766417e80a3a1)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:18.221153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39f36d-7886-47b1-bb6f-9de334981909 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:18.272087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bd39f36d-7886-47b1-bb6f-9de334981909 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6dc5d9dd1d24232a3e766417e80a3a1', 'request_id': 'req-bd39f36d-7886-47b1-bb6f-9de334981909'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:18.277600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bd39f36d-7886-47b1-bb6f-9de334981909 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6dc5d9dd1d24232a3e766417e80a3a1', 'request_id': 'req-bd39f36d-7886-47b1-bb6f-9de334981909'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:18.283845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bd39f36d-7886-47b1-bb6f-9de334981909 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6dc5d9dd1d24232a3e766417e80a3a1', 'request_id': 'req-bd39f36d-7886-47b1-bb6f-9de334981909'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:18.299395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bd39f36d-7886-47b1-bb6f-9de334981909 admin admin] The token cache is being invalidate because project c6dc5d9dd1d24232a3e766417e80a3a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:18.299785 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bd39f36d-7886-47b1-bb6f-9de334981909 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:18.301114 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bd39f36d-7886-47b1-bb6f-9de334981909 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:18.302722 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1301/2606] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:18 2022] DELETE /identity/v3/projects/c6dc5d9dd1d24232a3e766417e80a3a1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:18.308012 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8bee3-7cc6-466e-86e6-3cf47b47107b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:18.308514 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8bee3-7cc6-466e-86e6-3cf47b47107b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:18.308619 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8bee3-7cc6-466e-86e6-3cf47b47107b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:18.329601 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a9f8bee3-7cc6-466e-86e6-3cf47b47107b None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:18.330817 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a9f8bee3-7cc6-466e-86e6-3cf47b47107b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:18.397081 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1306/2607] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:56:18.405331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00cadd-fb27-4efd-bbc2-f4aba381033d None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:18.405612 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00cadd-fb27-4efd-bbc2-f4aba381033d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:18.405880 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00cadd-fb27-4efd-bbc2-f4aba381033d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:18.427921 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-1e00cadd-fb27-4efd-bbc2-f4aba381033d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:18.429183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-1e00cadd-fb27-4efd-bbc2-f4aba381033d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:18.438088 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1302/2608] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:18.442850 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8bee3-7cc6-466e-86e6-3cf47b47107b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:18.453571 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8bee3-7cc6-466e-86e6-3cf47b47107b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:18.458978 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05376235-90ab-468b-9c21-5f4b38cd41b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:18.459618 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05376235-90ab-468b-9c21-5f4b38cd41b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:18.459896 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05376235-90ab-468b-9c21-5f4b38cd41b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:18.460197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05376235-90ab-468b-9c21-5f4b38cd41b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:18.460986 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05376235-90ab-468b-9c21-5f4b38cd41b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:18.461289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05376235-90ab-468b-9c21-5f4b38cd41b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:18.462964 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05376235-90ab-468b-9c21-5f4b38cd41b0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:18.475311 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1307/2609] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:18.479759 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00cadd-fb27-4efd-bbc2-f4aba381033d None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:18.489270 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00cadd-fb27-4efd-bbc2-f4aba381033d None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:18.494187 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dcf819-b12c-4929-a355-5385fd96086a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:18.494781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dcf819-b12c-4929-a355-5385fd96086a admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:18.495053 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dcf819-b12c-4929-a355-5385fd96086a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:18.495338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dcf819-b12c-4929-a355-5385fd96086a admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:18.495954 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dcf819-b12c-4929-a355-5385fd96086a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:18.497874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dcf819-b12c-4929-a355-5385fd96086a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:18.526832 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1303/2610] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:18 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:56:18.531159 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05376235-90ab-468b-9c21-5f4b38cd41b0 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:18.541769 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05376235-90ab-468b-9c21-5f4b38cd41b0 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:18.611412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed7897-8626-42ef-ad14-3091d95727dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:18.615258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed7897-8626-42ef-ad14-3091d95727dd admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:18.615258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed7897-8626-42ef-ad14-3091d95727dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:18.615258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed7897-8626-42ef-ad14-3091d95727dd admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:18.615258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed7897-8626-42ef-ad14-3091d95727dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:18.615258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed7897-8626-42ef-ad14-3091d95727dd admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:18.642381 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:18.652959 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1308/2611] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:18 2022] POST /identity/v3/users => generated 300 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:18.653381 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dcf819-b12c-4929-a355-5385fd96086a admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:18.664449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dcf819-b12c-4929-a355-5385fd96086a admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:18.669827 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12048ef-fdf2-4326-97db-a92168481590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:18.670514 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12048ef-fdf2-4326-97db-a92168481590 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:18.670856 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12048ef-fdf2-4326-97db-a92168481590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:18.671184 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12048ef-fdf2-4326-97db-a92168481590 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:18.671919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12048ef-fdf2-4326-97db-a92168481590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:18.673594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12048ef-fdf2-4326-97db-a92168481590 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:18.685091 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1304/2612] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:56:18.689845 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed7897-8626-42ef-ad14-3091d95727dd admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:18.701346 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed7897-8626-42ef-ad14-3091d95727dd admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:18.706932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f12e8-858d-48d3-ba80-c9a89ca5c9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:18.707602 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f12e8-858d-48d3-ba80-c9a89ca5c9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:18.707923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f12e8-858d-48d3-ba80-c9a89ca5c9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:18.708282 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f12e8-858d-48d3-ba80-c9a89ca5c9a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:18.709033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f12e8-858d-48d3-ba80-c9a89ca5c9a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:18.710709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f12e8-858d-48d3-ba80-c9a89ca5c9a1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:18.722016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1309/2613] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:18.726847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12048ef-fdf2-4326-97db-a92168481590 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:18.746403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12048ef-fdf2-4326-97db-a92168481590 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:18.746403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956fbae8-ab02-44f1-a91b-fcf5bf09858d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:18.746403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956fbae8-ab02-44f1-a91b-fcf5bf09858d admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:18.746403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956fbae8-ab02-44f1-a91b-fcf5bf09858d admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:18.746823 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956fbae8-ab02-44f1-a91b-fcf5bf09858d admin admin] PATH_INFO: `/v3/projects/a663360806e041e0a6adfb97b92e3b33/users/6630cf0f761b426c88b6d95e807c7b3c/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:18.763754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956fbae8-ab02-44f1-a91b-fcf5bf09858d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a663360806e041e0a6adfb97b92e3b33, user_id=6630cf0f761b426c88b6d95e807c7b3c, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:18.767735 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956fbae8-ab02-44f1-a91b-fcf5bf09858d admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:18.787229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1305/2614] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:18 2022] PUT /identity/v3/projects/a663360806e041e0a6adfb97b92e3b33/users/6630cf0f761b426c88b6d95e807c7b3c/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:18.799123 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26c98e-ad69-4f3a-9ba3-0a42f55bde34 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:18.799403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26c98e-ad69-4f3a-9ba3-0a42f55bde34 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:18.799667 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26c98e-ad69-4f3a-9ba3-0a42f55bde34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:18.830526 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9c26c98e-ad69-4f3a-9ba3-0a42f55bde34 None None] MFA Rules not processed for user `6630cf0f761b426c88b6d95e807c7b3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:18.831733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9c26c98e-ad69-4f3a-9ba3-0a42f55bde34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:18.902461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1310/2615] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:18.914251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956fbae8-ab02-44f1-a91b-fcf5bf09858d admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:18.916128 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-956fbae8-ab02-44f1-a91b-fcf5bf09858d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:18.927228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:18.977592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956fbae8-ab02-44f1-a91b-fcf5bf09858d admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:18.981992 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28bb21-4bd1-46af-8999-ed6acabb674e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:18.982606 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28bb21-4bd1-46af-8999-ed6acabb674e service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:18.982857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28bb21-4bd1-46af-8999-ed6acabb674e service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:18.983135 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28bb21-4bd1-46af-8999-ed6acabb674e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:18.992676 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28bb21-4bd1-46af-8999-ed6acabb674e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:18.994107 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28bb21-4bd1-46af-8999-ed6acabb674e service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:19.003236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1306/2616] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:18 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:19.859938 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26c98e-ad69-4f3a-9ba3-0a42f55bde34 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:19.861495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9c26c98e-ad69-4f3a-9ba3-0a42f55bde34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:19.923263 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26c98e-ad69-4f3a-9ba3-0a42f55bde34 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:19.928375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7bf1cc-d35e-40e4-8d29-8e00fea313c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:19.928985 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bf1cc-d35e-40e4-8d29-8e00fea313c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:19.929267 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bf1cc-d35e-40e4-8d29-8e00fea313c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:19.929531 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bf1cc-d35e-40e4-8d29-8e00fea313c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:19.931832 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0e7bf1cc-d35e-40e4-8d29-8e00fea313c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:20.012142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7bf1cc-d35e-40e4-8d29-8e00fea313c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:20.013694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7bf1cc-d35e-40e4-8d29-8e00fea313c4 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:20.024413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1311/2617] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:23.181229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28bb21-4bd1-46af-8999-ed6acabb674e service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:23.182699 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-bc28bb21-4bd1-46af-8999-ed6acabb674e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:23.197976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28bb21-4bd1-46af-8999-ed6acabb674e service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:23.202674 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba13148-c7ac-41aa-999f-69d48fde7a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:23.203324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba13148-c7ac-41aa-999f-69d48fde7a88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:23.203601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba13148-c7ac-41aa-999f-69d48fde7a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:23.203867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba13148-c7ac-41aa-999f-69d48fde7a88 admin admin] PATH_INFO: `/v3/users/521cc5ea8fb045bc81e8e3bfa8ba239b` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:23.205105 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba13148-c7ac-41aa-999f-69d48fde7a88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=521cc5ea8fb045bc81e8e3bfa8ba239b)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:23.207029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba13148-c7ac-41aa-999f-69d48fde7a88 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:23.271212 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6ba13148-c7ac-41aa-999f-69d48fde7a88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '521cc5ea8fb045bc81e8e3bfa8ba239b'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:23.271921 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6ba13148-c7ac-41aa-999f-69d48fde7a88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '521cc5ea8fb045bc81e8e3bfa8ba239b'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:23.272580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6ba13148-c7ac-41aa-999f-69d48fde7a88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '521cc5ea8fb045bc81e8e3bfa8ba239b'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:23.282027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6ba13148-c7ac-41aa-999f-69d48fde7a88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '521cc5ea8fb045bc81e8e3bfa8ba239b'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:23.295930 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6ba13148-c7ac-41aa-999f-69d48fde7a88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '521cc5ea8fb045bc81e8e3bfa8ba239b'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:23.313722 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1307/2618] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:23 2022] DELETE /identity/v3/users/521cc5ea8fb045bc81e8e3bfa8ba239b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:23.524189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7bf1cc-d35e-40e4-8d29-8e00fea313c4 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:23.534771 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7bf1cc-d35e-40e4-8d29-8e00fea313c4 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:23.563414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:23.595565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bcc7f-1443-49e0-938a-6fbf1aab4aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:23.596225 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bcc7f-1443-49e0-938a-6fbf1aab4aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:23.596502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bcc7f-1443-49e0-938a-6fbf1aab4aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:23.596790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bcc7f-1443-49e0-938a-6fbf1aab4aa8 admin admin] PATH_INFO: `/v3/projects/21f6e5be24e448f9b981ad77d23de8bd` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:23.598336 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bcc7f-1443-49e0-938a-6fbf1aab4aa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21f6e5be24e448f9b981ad77d23de8bd)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:23.600258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bcc7f-1443-49e0-938a-6fbf1aab4aa8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:23.649289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4f9bcc7f-1443-49e0-938a-6fbf1aab4aa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21f6e5be24e448f9b981ad77d23de8bd', 'request_id': 'req-4f9bcc7f-1443-49e0-938a-6fbf1aab4aa8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:23.658402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4f9bcc7f-1443-49e0-938a-6fbf1aab4aa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21f6e5be24e448f9b981ad77d23de8bd', 'request_id': 'req-4f9bcc7f-1443-49e0-938a-6fbf1aab4aa8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:23.664731 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4f9bcc7f-1443-49e0-938a-6fbf1aab4aa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21f6e5be24e448f9b981ad77d23de8bd', 'request_id': 'req-4f9bcc7f-1443-49e0-938a-6fbf1aab4aa8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:23.676161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4f9bcc7f-1443-49e0-938a-6fbf1aab4aa8 admin admin] The token cache is being invalidate because project 21f6e5be24e448f9b981ad77d23de8bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:23.676452 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4f9bcc7f-1443-49e0-938a-6fbf1aab4aa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:23.677086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4f9bcc7f-1443-49e0-938a-6fbf1aab4aa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:23.678589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1312/2619] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:23 2022] DELETE /identity/v3/projects/21f6e5be24e448f9b981ad77d23de8bd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:23.683874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8f14b-23cf-45ae-aa8e-022a045e2576 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:23.684137 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8f14b-23cf-45ae-aa8e-022a045e2576 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:23.684432 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8f14b-23cf-45ae-aa8e-022a045e2576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:23.706593 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-9bf8f14b-23cf-45ae-aa8e-022a045e2576 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:23.707908 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-9bf8f14b-23cf-45ae-aa8e-022a045e2576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:23.770890 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1308/2620] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:56:23.778968 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e84d3-449c-4ef6-bb93-01cd7d7d7333 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:23.779256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e84d3-449c-4ef6-bb93-01cd7d7d7333 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:23.779529 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e84d3-449c-4ef6-bb93-01cd7d7d7333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:23.802399 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2b3e84d3-449c-4ef6-bb93-01cd7d7d7333 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:23.803632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2b3e84d3-449c-4ef6-bb93-01cd7d7d7333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:23.815865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1313/2621] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:23.822286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8f14b-23cf-45ae-aa8e-022a045e2576 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:23.835610 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8f14b-23cf-45ae-aa8e-022a045e2576 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:23.840663 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e6b03-6d6f-4adc-bf7a-1ba572f488e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:23.841289 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e6b03-6d6f-4adc-bf7a-1ba572f488e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:23.841545 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e6b03-6d6f-4adc-bf7a-1ba572f488e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:23.841809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e6b03-6d6f-4adc-bf7a-1ba572f488e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:23.842463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e6b03-6d6f-4adc-bf7a-1ba572f488e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:23.842761 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e6b03-6d6f-4adc-bf7a-1ba572f488e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:23.844407 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e6b03-6d6f-4adc-bf7a-1ba572f488e2 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:23.857034 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1309/2622] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:23.861671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e84d3-449c-4ef6-bb93-01cd7d7d7333 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:23.871641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e84d3-449c-4ef6-bb93-01cd7d7d7333 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:23.877258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554f563-9ef1-4fb9-8346-1018b259fb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:23.877993 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554f563-9ef1-4fb9-8346-1018b259fb06 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:23.878175 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554f563-9ef1-4fb9-8346-1018b259fb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:23.878403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554f563-9ef1-4fb9-8346-1018b259fb06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:23.879034 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554f563-9ef1-4fb9-8346-1018b259fb06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:23.881150 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554f563-9ef1-4fb9-8346-1018b259fb06 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:23.911724 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1314/2623] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:23 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:23.917771 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e6b03-6d6f-4adc-bf7a-1ba572f488e2 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:23.929647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e6b03-6d6f-4adc-bf7a-1ba572f488e2 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:23.934833 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:23.993246 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad65c0-6814-48c0-a0ee-9acfa2ddc928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:23.993853 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad65c0-6814-48c0-a0ee-9acfa2ddc928 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:23.994130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad65c0-6814-48c0-a0ee-9acfa2ddc928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:23.994395 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad65c0-6814-48c0-a0ee-9acfa2ddc928 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:23.995043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad65c0-6814-48c0-a0ee-9acfa2ddc928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:23.997008 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad65c0-6814-48c0-a0ee-9acfa2ddc928 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.022770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1310/2624] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:23 2022] POST /identity/v3/users => generated 302 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:56:24.027075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554f563-9ef1-4fb9-8346-1018b259fb06 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.037822 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554f563-9ef1-4fb9-8346-1018b259fb06 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.042488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27180cbd-58bf-4ab3-92ad-827b68e66a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.043089 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27180cbd-58bf-4ab3-92ad-827b68e66a26 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.043359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27180cbd-58bf-4ab3-92ad-827b68e66a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.043643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27180cbd-58bf-4ab3-92ad-827b68e66a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.044315 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27180cbd-58bf-4ab3-92ad-827b68e66a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.045913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27180cbd-58bf-4ab3-92ad-827b68e66a26 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.056409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1315/2625] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:24.060581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad65c0-6814-48c0-a0ee-9acfa2ddc928 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.073907 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad65c0-6814-48c0-a0ee-9acfa2ddc928 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.079688 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863ea5f-3039-45ff-b1e4-7c3341d59305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.080488 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863ea5f-3039-45ff-b1e4-7c3341d59305 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.080874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863ea5f-3039-45ff-b1e4-7c3341d59305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.081267 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863ea5f-3039-45ff-b1e4-7c3341d59305 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.082022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863ea5f-3039-45ff-b1e4-7c3341d59305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.083833 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863ea5f-3039-45ff-b1e4-7c3341d59305 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.096433 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1311/2626] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:24.101162 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27180cbd-58bf-4ab3-92ad-827b68e66a26 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.112825 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27180cbd-58bf-4ab3-92ad-827b68e66a26 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.118464 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc2e25-fc3c-4e4b-ab13-3ed65a2e5f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.119249 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc2e25-fc3c-4e4b-ab13-3ed65a2e5f66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.119599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc2e25-fc3c-4e4b-ab13-3ed65a2e5f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.119928 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc2e25-fc3c-4e4b-ab13-3ed65a2e5f66 admin admin] PATH_INFO: `/v3/projects/801560474fd644209a7b6332c7d57fd7/users/b74b624a964e4e128435a2dca3063526/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.138112 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc2e25-fc3c-4e4b-ab13-3ed65a2e5f66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=801560474fd644209a7b6332c7d57fd7, user_id=b74b624a964e4e128435a2dca3063526, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.140916 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc2e25-fc3c-4e4b-ab13-3ed65a2e5f66 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.157751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1316/2627] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:24 2022] PUT /identity/v3/projects/801560474fd644209a7b6332c7d57fd7/users/b74b624a964e4e128435a2dca3063526/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:56:24.170493 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fbb66-28c7-493d-837d-4ef846fc647a None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.170900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fbb66-28c7-493d-837d-4ef846fc647a None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.171296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fbb66-28c7-493d-837d-4ef846fc647a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.202410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-008fbb66-28c7-493d-837d-4ef846fc647a None None] MFA Rules not processed for user `b74b624a964e4e128435a2dca3063526`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:24.203601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-008fbb66-28c7-493d-837d-4ef846fc647a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:24.274054 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1312/2628] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:24.278658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc2e25-fc3c-4e4b-ab13-3ed65a2e5f66 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.348219 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:24.356525 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc2e25-fc3c-4e4b-ab13-3ed65a2e5f66 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.365641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b83d5a-cdd8-4304-8d0e-1a37296aa2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.366627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b83d5a-cdd8-4304-8d0e-1a37296aa2ad admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.366914 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b83d5a-cdd8-4304-8d0e-1a37296aa2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.367362 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b83d5a-cdd8-4304-8d0e-1a37296aa2ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.368056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b83d5a-cdd8-4304-8d0e-1a37296aa2ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.370270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b83d5a-cdd8-4304-8d0e-1a37296aa2ad admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.402191 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1317/2629] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/projects => generated 403 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:24.408585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fbb66-28c7-493d-837d-4ef846fc647a None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.425292 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fbb66-28c7-493d-837d-4ef846fc647a None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.487986 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fb1ac-c0d3-4b9a-b501-b81454d18315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.488572 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fb1ac-c0d3-4b9a-b501-b81454d18315 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.488866 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fb1ac-c0d3-4b9a-b501-b81454d18315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.489140 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fb1ac-c0d3-4b9a-b501-b81454d18315 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.489781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fb1ac-c0d3-4b9a-b501-b81454d18315 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.491714 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fb1ac-c0d3-4b9a-b501-b81454d18315 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.523790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1313/2630] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/users => generated 308 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:24.529633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b83d5a-cdd8-4304-8d0e-1a37296aa2ad admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.545644 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b83d5a-cdd8-4304-8d0e-1a37296aa2ad admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.553501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0d104-2e0b-4723-9282-b76c16cbbe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.554119 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0d104-2e0b-4723-9282-b76c16cbbe8e admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.554398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0d104-2e0b-4723-9282-b76c16cbbe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.554667 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0d104-2e0b-4723-9282-b76c16cbbe8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.555362 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0d104-2e0b-4723-9282-b76c16cbbe8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.556999 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0d104-2e0b-4723-9282-b76c16cbbe8e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.568984 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1318/2631] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:24.573674 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fb1ac-c0d3-4b9a-b501-b81454d18315 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.585246 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fb1ac-c0d3-4b9a-b501-b81454d18315 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.590202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8d170-90f8-4e7a-bcf1-eb2ae79e2798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.590866 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8d170-90f8-4e7a-bcf1-eb2ae79e2798 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.591139 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8d170-90f8-4e7a-bcf1-eb2ae79e2798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.591404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8d170-90f8-4e7a-bcf1-eb2ae79e2798 admin admin] PATH_INFO: `/v3/projects/399f5dba7c964cbb899f78f3279d765f/users/a9341ec6e4fe41f1837f7a72511c8e01/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.593768 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:24.610585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8d170-90f8-4e7a-bcf1-eb2ae79e2798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=399f5dba7c964cbb899f78f3279d765f, user_id=a9341ec6e4fe41f1837f7a72511c8e01, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.613312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8d170-90f8-4e7a-bcf1-eb2ae79e2798 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.629643 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1314/2632] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:24 2022] PUT /identity/v3/projects/399f5dba7c964cbb899f78f3279d765f/users/a9341ec6e4fe41f1837f7a72511c8e01/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:24.633942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0d104-2e0b-4723-9282-b76c16cbbe8e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.645764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0d104-2e0b-4723-9282-b76c16cbbe8e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.708546 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0ec93-b8d6-4ca8-aae1-5f89a874db98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.709147 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0ec93-b8d6-4ca8-aae1-5f89a874db98 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.709496 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0ec93-b8d6-4ca8-aae1-5f89a874db98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.709638 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0ec93-b8d6-4ca8-aae1-5f89a874db98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.710297 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0ec93-b8d6-4ca8-aae1-5f89a874db98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.711902 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0ec93-b8d6-4ca8-aae1-5f89a874db98 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.721830 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1319/2633] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:24.726376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8d170-90f8-4e7a-bcf1-eb2ae79e2798 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.737099 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8d170-90f8-4e7a-bcf1-eb2ae79e2798 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.743011 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32590b4a-f616-4726-baba-77b9e57ccf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.743602 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32590b4a-f616-4726-baba-77b9e57ccf1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.743880 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32590b4a-f616-4726-baba-77b9e57ccf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.744191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32590b4a-f616-4726-baba-77b9e57ccf1b admin admin] PATH_INFO: `/v3/projects/399f5dba7c964cbb899f78f3279d765f/users/a9341ec6e4fe41f1837f7a72511c8e01/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.746163 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32590b4a-f616-4726-baba-77b9e57ccf1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=399f5dba7c964cbb899f78f3279d765f, user_id=a9341ec6e4fe41f1837f7a72511c8e01, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.748853 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32590b4a-f616-4726-baba-77b9e57ccf1b admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.760234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1315/2634] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:24 2022] PUT /identity/v3/projects/399f5dba7c964cbb899f78f3279d765f/users/a9341ec6e4fe41f1837f7a72511c8e01/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:24.771408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43bfeb-15f8-4620-adc8-fdec34580d0e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.771645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43bfeb-15f8-4620-adc8-fdec34580d0e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.772016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43bfeb-15f8-4620-adc8-fdec34580d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.807875 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-bf43bfeb-15f8-4620-adc8-fdec34580d0e None None] MFA Rules not processed for user `a9341ec6e4fe41f1837f7a72511c8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:24.809133 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bf43bfeb-15f8-4620-adc8-fdec34580d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:24.894642 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1320/2635] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:24.911278 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32590b4a-f616-4726-baba-77b9e57ccf1b admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.913140 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-32590b4a-f616-4726-baba-77b9e57ccf1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:24.977524 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32590b4a-f616-4726-baba-77b9e57ccf1b admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.981972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095185f3-60a3-4e53-b733-f05032ec03a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.982200 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095185f3-60a3-4e53-b733-f05032ec03a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.982468 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095185f3-60a3-4e53-b733-f05032ec03a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.982726 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095185f3-60a3-4e53-b733-f05032ec03a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.992300 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095185f3-60a3-4e53-b733-f05032ec03a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.993828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095185f3-60a3-4e53-b733-f05032ec03a7 service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:25.003064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1316/2636] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:24 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:25.232741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43bfeb-15f8-4620-adc8-fdec34580d0e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:25.309202 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:25.316077 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43bfeb-15f8-4620-adc8-fdec34580d0e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:25.322537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890fd8a-3516-472a-ba95-41af3dd9a8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:25.323196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890fd8a-3516-472a-ba95-41af3dd9a8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:25.323472 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890fd8a-3516-472a-ba95-41af3dd9a8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:25.323739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890fd8a-3516-472a-ba95-41af3dd9a8d6 admin admin] PATH_INFO: `/v3/users/b74b624a964e4e128435a2dca3063526` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:25.324904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890fd8a-3516-472a-ba95-41af3dd9a8d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b74b624a964e4e128435a2dca3063526)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:25.326833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890fd8a-3516-472a-ba95-41af3dd9a8d6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:25.388313 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8890fd8a-3516-472a-ba95-41af3dd9a8d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b74b624a964e4e128435a2dca3063526'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:25.395775 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8890fd8a-3516-472a-ba95-41af3dd9a8d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b74b624a964e4e128435a2dca3063526'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:25.404340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8890fd8a-3516-472a-ba95-41af3dd9a8d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b74b624a964e4e128435a2dca3063526'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:25.405281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8890fd8a-3516-472a-ba95-41af3dd9a8d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b74b624a964e4e128435a2dca3063526'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:25.423207 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8890fd8a-3516-472a-ba95-41af3dd9a8d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b74b624a964e4e128435a2dca3063526'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:25.432873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1321/2637] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:25 2022] DELETE /identity/v3/users/b74b624a964e4e128435a2dca3063526 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:25.445207 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629b133-2e1c-424d-988f-d31d724a7f3e None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:25.445771 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629b133-2e1c-424d-988f-d31d724a7f3e None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:25.445771 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629b133-2e1c-424d-988f-d31d724a7f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:25.467638 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a629b133-2e1c-424d-988f-d31d724a7f3e None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:25.469131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a629b133-2e1c-424d-988f-d31d724a7f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:25.537822 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1317/2638] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:25.550365 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890fd8a-3516-472a-ba95-41af3dd9a8d6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:25.552100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8890fd8a-3516-472a-ba95-41af3dd9a8d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:25.616886 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890fd8a-3516-472a-ba95-41af3dd9a8d6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:25.620751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654974fa-0767-439b-9874-7d83a0015e93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:25.621371 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654974fa-0767-439b-9874-7d83a0015e93 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:25.621649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654974fa-0767-439b-9874-7d83a0015e93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:25.621921 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654974fa-0767-439b-9874-7d83a0015e93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:25.633578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654974fa-0767-439b-9874-7d83a0015e93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:25.635159 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654974fa-0767-439b-9874-7d83a0015e93 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:25.645848 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1322/2639] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:26.237583 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629b133-2e1c-424d-988f-d31d724a7f3e None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.239158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a629b133-2e1c-424d-988f-d31d724a7f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:26.251565 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629b133-2e1c-424d-988f-d31d724a7f3e None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.258000 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621b05d-77ab-45c5-bde1-9602efe633ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.258668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621b05d-77ab-45c5-bde1-9602efe633ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.258929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621b05d-77ab-45c5-bde1-9602efe633ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.259259 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621b05d-77ab-45c5-bde1-9602efe633ae admin admin] PATH_INFO: `/v3/projects/801560474fd644209a7b6332c7d57fd7` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.260556 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621b05d-77ab-45c5-bde1-9602efe633ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=801560474fd644209a7b6332c7d57fd7)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.262498 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621b05d-77ab-45c5-bde1-9602efe633ae admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.322297 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-7621b05d-77ab-45c5-bde1-9602efe633ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '801560474fd644209a7b6332c7d57fd7', 'request_id': 'req-7621b05d-77ab-45c5-bde1-9602efe633ae'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:26.329001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-7621b05d-77ab-45c5-bde1-9602efe633ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '801560474fd644209a7b6332c7d57fd7', 'request_id': 'req-7621b05d-77ab-45c5-bde1-9602efe633ae'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:26.336696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-7621b05d-77ab-45c5-bde1-9602efe633ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '801560474fd644209a7b6332c7d57fd7', 'request_id': 'req-7621b05d-77ab-45c5-bde1-9602efe633ae'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:26.340619 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:26.353823 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-7621b05d-77ab-45c5-bde1-9602efe633ae admin admin] The token cache is being invalidate because project 801560474fd644209a7b6332c7d57fd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:26.354125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-7621b05d-77ab-45c5-bde1-9602efe633ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:26.354759 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-7621b05d-77ab-45c5-bde1-9602efe633ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:26.356337 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1318/2640] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:26 2022] DELETE /identity/v3/projects/801560474fd644209a7b6332c7d57fd7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:26.361274 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654974fa-0767-439b-9874-7d83a0015e93 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.372161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654974fa-0767-439b-9874-7d83a0015e93 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.435202 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9a234-cd37-4004-9b7a-b525a3cd68f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.435738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9a234-cd37-4004-9b7a-b525a3cd68f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.436034 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9a234-cd37-4004-9b7a-b525a3cd68f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.436303 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9a234-cd37-4004-9b7a-b525a3cd68f6 admin admin] PATH_INFO: `/v3/users/a9341ec6e4fe41f1837f7a72511c8e01` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.437447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9a234-cd37-4004-9b7a-b525a3cd68f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9341ec6e4fe41f1837f7a72511c8e01)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.439365 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9a234-cd37-4004-9b7a-b525a3cd68f6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.455415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:26.494089 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-67e9a234-cd37-4004-9b7a-b525a3cd68f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9341ec6e4fe41f1837f7a72511c8e01'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:26.501577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-67e9a234-cd37-4004-9b7a-b525a3cd68f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9341ec6e4fe41f1837f7a72511c8e01'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:26.509982 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-67e9a234-cd37-4004-9b7a-b525a3cd68f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9341ec6e4fe41f1837f7a72511c8e01'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:26.516208 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-67e9a234-cd37-4004-9b7a-b525a3cd68f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9341ec6e4fe41f1837f7a72511c8e01'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:26.533337 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-67e9a234-cd37-4004-9b7a-b525a3cd68f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9341ec6e4fe41f1837f7a72511c8e01'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:26.540456 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1323/2641] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:26 2022] DELETE /identity/v3/users/a9341ec6e4fe41f1837f7a72511c8e01 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:26.836136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621b05d-77ab-45c5-bde1-9602efe633ae admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.837687 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7621b05d-77ab-45c5-bde1-9602efe633ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:26.851161 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621b05d-77ab-45c5-bde1-9602efe633ae admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.914637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a5613-50f9-4337-a153-3094e40aaf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.915305 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a5613-50f9-4337-a153-3094e40aaf0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.915560 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a5613-50f9-4337-a153-3094e40aaf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.915833 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a5613-50f9-4337-a153-3094e40aaf0a admin admin] PATH_INFO: `/v3/projects/399f5dba7c964cbb899f78f3279d765f` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.917091 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a5613-50f9-4337-a153-3094e40aaf0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=399f5dba7c964cbb899f78f3279d765f)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.919049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a5613-50f9-4337-a153-3094e40aaf0a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.979819 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-253a5613-50f9-4337-a153-3094e40aaf0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399f5dba7c964cbb899f78f3279d765f', 'request_id': 'req-253a5613-50f9-4337-a153-3094e40aaf0a'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:26.986253 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-253a5613-50f9-4337-a153-3094e40aaf0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399f5dba7c964cbb899f78f3279d765f', 'request_id': 'req-253a5613-50f9-4337-a153-3094e40aaf0a'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:26.993348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-253a5613-50f9-4337-a153-3094e40aaf0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399f5dba7c964cbb899f78f3279d765f', 'request_id': 'req-253a5613-50f9-4337-a153-3094e40aaf0a'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:27.008137 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-253a5613-50f9-4337-a153-3094e40aaf0a admin admin] The token cache is being invalidate because project 399f5dba7c964cbb899f78f3279d765f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:27.008426 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-253a5613-50f9-4337-a153-3094e40aaf0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:27.009036 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-253a5613-50f9-4337-a153-3094e40aaf0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:27.010749 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1319/2642] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:26 2022] DELETE /identity/v3/projects/399f5dba7c964cbb899f78f3279d765f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:27.015746 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fbd25-66bb-47e7-af05-0dc697f6a80f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.016250 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fbd25-66bb-47e7-af05-0dc697f6a80f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.016358 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fbd25-66bb-47e7-af05-0dc697f6a80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.037341 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a99fbd25-66bb-47e7-af05-0dc697f6a80f None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:27.038568 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a99fbd25-66bb-47e7-af05-0dc697f6a80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:27.106936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1324/2643] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:27.114825 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9941ef-82f7-4bcb-8811-75be545a7b85 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.115109 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9941ef-82f7-4bcb-8811-75be545a7b85 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.115399 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9941ef-82f7-4bcb-8811-75be545a7b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.136037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-6a9941ef-82f7-4bcb-8811-75be545a7b85 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:27.137307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-6a9941ef-82f7-4bcb-8811-75be545a7b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:27.147780 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1320/2644] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:27.152460 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fbd25-66bb-47e7-af05-0dc697f6a80f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.163704 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fbd25-66bb-47e7-af05-0dc697f6a80f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.169523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308f786-1ef0-4b2f-8c27-631d3baebcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.170125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308f786-1ef0-4b2f-8c27-631d3baebcda admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.170398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308f786-1ef0-4b2f-8c27-631d3baebcda admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.170659 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308f786-1ef0-4b2f-8c27-631d3baebcda admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.171300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308f786-1ef0-4b2f-8c27-631d3baebcda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:27.171575 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308f786-1ef0-4b2f-8c27-631d3baebcda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.173204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308f786-1ef0-4b2f-8c27-631d3baebcda admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.186891 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1325/2645] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:27.191463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9941ef-82f7-4bcb-8811-75be545a7b85 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.201147 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9941ef-82f7-4bcb-8811-75be545a7b85 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.206191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a5df8-d00c-4328-bfbb-d878b2cb5bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.206791 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a5df8-d00c-4328-bfbb-d878b2cb5bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.207059 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a5df8-d00c-4328-bfbb-d878b2cb5bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.207330 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a5df8-d00c-4328-bfbb-d878b2cb5bb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.208035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a5df8-d00c-4328-bfbb-d878b2cb5bb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.210035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a5df8-d00c-4328-bfbb-d878b2cb5bb4 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.244477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1321/2646] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:27.245853 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308f786-1ef0-4b2f-8c27-631d3baebcda admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.256841 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308f786-1ef0-4b2f-8c27-631d3baebcda admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.320808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f243fb-4c00-4468-814c-f41c24cb8ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.321428 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f243fb-4c00-4468-814c-f41c24cb8ddb admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.321704 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f243fb-4c00-4468-814c-f41c24cb8ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.321973 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f243fb-4c00-4468-814c-f41c24cb8ddb admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.322604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f243fb-4c00-4468-814c-f41c24cb8ddb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.324601 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f243fb-4c00-4468-814c-f41c24cb8ddb admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.349480 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1326/2647] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:27.353776 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a5df8-d00c-4328-bfbb-d878b2cb5bb4 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.365107 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a5df8-d00c-4328-bfbb-d878b2cb5bb4 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.370700 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383e061-54ea-48af-80ab-0424c478fb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.371326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383e061-54ea-48af-80ab-0424c478fb38 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.371592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383e061-54ea-48af-80ab-0424c478fb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.371879 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383e061-54ea-48af-80ab-0424c478fb38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.372586 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383e061-54ea-48af-80ab-0424c478fb38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.374217 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383e061-54ea-48af-80ab-0424c478fb38 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.384853 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1322/2648] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:27.389066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f243fb-4c00-4468-814c-f41c24cb8ddb admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.399107 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f243fb-4c00-4468-814c-f41c24cb8ddb admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.404025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a354ca1-84c2-47f8-8652-e1e9b663b467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.404622 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a354ca1-84c2-47f8-8652-e1e9b663b467 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.404890 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a354ca1-84c2-47f8-8652-e1e9b663b467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.405160 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a354ca1-84c2-47f8-8652-e1e9b663b467 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.405798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a354ca1-84c2-47f8-8652-e1e9b663b467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.407387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a354ca1-84c2-47f8-8652-e1e9b663b467 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.409662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:27.419992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1327/2649] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:27.424156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383e061-54ea-48af-80ab-0424c478fb38 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.435196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383e061-54ea-48af-80ab-0424c478fb38 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.440699 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7044d6a-8af0-4284-8768-c027d26caadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.441358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7044d6a-8af0-4284-8768-c027d26caadf admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.441695 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7044d6a-8af0-4284-8768-c027d26caadf admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.441906 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7044d6a-8af0-4284-8768-c027d26caadf admin admin] PATH_INFO: `/v3/projects/260581f7676c4a24bb054fb3cf3228b5/users/41f1307f33a1461eb165b157845fa3f7/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.460079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7044d6a-8af0-4284-8768-c027d26caadf admin admin] RBAC: Authorizing `identity:create_grant(project_id=260581f7676c4a24bb054fb3cf3228b5, user_id=41f1307f33a1461eb165b157845fa3f7, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.462874 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7044d6a-8af0-4284-8768-c027d26caadf admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.477329 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1323/2650] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:27 2022] PUT /identity/v3/projects/260581f7676c4a24bb054fb3cf3228b5/users/41f1307f33a1461eb165b157845fa3f7/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:27.490045 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42f9c2-4b33-4d25-8004-5245dd7fce20 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.490820 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42f9c2-4b33-4d25-8004-5245dd7fce20 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.491125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42f9c2-4b33-4d25-8004-5245dd7fce20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.521882 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1e42f9c2-4b33-4d25-8004-5245dd7fce20 None None] MFA Rules not processed for user `41f1307f33a1461eb165b157845fa3f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:27.523110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1e42f9c2-4b33-4d25-8004-5245dd7fce20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:27.593363 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1328/2651] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:27.598396 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7044d6a-8af0-4284-8768-c027d26caadf admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.627627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:27.702637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7044d6a-8af0-4284-8768-c027d26caadf admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.707604 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a5853-b0d1-4e01-8584-26b1f36d3622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.708258 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a5853-b0d1-4e01-8584-26b1f36d3622 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.708617 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a5853-b0d1-4e01-8584-26b1f36d3622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.708882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a5853-b0d1-4e01-8584-26b1f36d3622 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.709634 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a5853-b0d1-4e01-8584-26b1f36d3622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.711647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a5853-b0d1-4e01-8584-26b1f36d3622 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.741338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1324/2652] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/projects => generated 387 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:56:27.745541 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42f9c2-4b33-4d25-8004-5245dd7fce20 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.757161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42f9c2-4b33-4d25-8004-5245dd7fce20 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.824171 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f4eb1-928e-4847-9fcb-c18cbdaf26dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.824810 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f4eb1-928e-4847-9fcb-c18cbdaf26dd admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.825092 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f4eb1-928e-4847-9fcb-c18cbdaf26dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.825368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f4eb1-928e-4847-9fcb-c18cbdaf26dd admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.825999 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f4eb1-928e-4847-9fcb-c18cbdaf26dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.828033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f4eb1-928e-4847-9fcb-c18cbdaf26dd admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.855961 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1329/2653] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:27.857374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a5853-b0d1-4e01-8584-26b1f36d3622 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.869264 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a5853-b0d1-4e01-8584-26b1f36d3622 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.874785 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ecc162-807e-4c71-8c67-dac29ba874c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.875409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecc162-807e-4c71-8c67-dac29ba874c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.875682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecc162-807e-4c71-8c67-dac29ba874c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.875944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecc162-807e-4c71-8c67-dac29ba874c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.876631 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ecc162-807e-4c71-8c67-dac29ba874c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.878250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ecc162-807e-4c71-8c67-dac29ba874c0 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.894164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1325/2654] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:27.895724 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f4eb1-928e-4847-9fcb-c18cbdaf26dd admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.906989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f4eb1-928e-4847-9fcb-c18cbdaf26dd admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.914000 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f2dbd-8aad-46df-b69c-89b233ed91af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.914835 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f2dbd-8aad-46df-b69c-89b233ed91af admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.915088 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f2dbd-8aad-46df-b69c-89b233ed91af admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.915379 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f2dbd-8aad-46df-b69c-89b233ed91af admin admin] PATH_INFO: `/v3/projects/8e8fcfa98e904cb0bceb1a360fb631b1/users/1f9f5086c41d407a8177298ec54126a4/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.933194 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f2dbd-8aad-46df-b69c-89b233ed91af admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e8fcfa98e904cb0bceb1a360fb631b1, user_id=1f9f5086c41d407a8177298ec54126a4, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.935880 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f2dbd-8aad-46df-b69c-89b233ed91af admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.950504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1330/2655] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:27 2022] PUT /identity/v3/projects/8e8fcfa98e904cb0bceb1a360fb631b1/users/1f9f5086c41d407a8177298ec54126a4/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:27.954696 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ecc162-807e-4c71-8c67-dac29ba874c0 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.966973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ecc162-807e-4c71-8c67-dac29ba874c0 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:28.032249 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad250d56-7dc4-41e4-a395-111f0656ed18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:28.032869 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad250d56-7dc4-41e4-a395-111f0656ed18 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.033143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad250d56-7dc4-41e4-a395-111f0656ed18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.033408 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad250d56-7dc4-41e4-a395-111f0656ed18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.034051 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad250d56-7dc4-41e4-a395-111f0656ed18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:28.035692 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad250d56-7dc4-41e4-a395-111f0656ed18 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:28.046409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1326/2656] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:28.051103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f2dbd-8aad-46df-b69c-89b233ed91af admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:28.062715 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f2dbd-8aad-46df-b69c-89b233ed91af admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:28.067734 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39465361-f739-4ad0-9b1d-37f0c8d092f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:28.068475 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39465361-f739-4ad0-9b1d-37f0c8d092f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.068976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39465361-f739-4ad0-9b1d-37f0c8d092f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.069320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39465361-f739-4ad0-9b1d-37f0c8d092f6 admin admin] PATH_INFO: `/v3/projects/8e8fcfa98e904cb0bceb1a360fb631b1/users/1f9f5086c41d407a8177298ec54126a4/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.071386 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39465361-f739-4ad0-9b1d-37f0c8d092f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e8fcfa98e904cb0bceb1a360fb631b1, user_id=1f9f5086c41d407a8177298ec54126a4, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:28.074056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39465361-f739-4ad0-9b1d-37f0c8d092f6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:28.086781 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1331/2657] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:28 2022] PUT /identity/v3/projects/8e8fcfa98e904cb0bceb1a360fb631b1/users/1f9f5086c41d407a8177298ec54126a4/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:28.097678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff005d4-e21b-41c2-b860-a6e4e98df109 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.097969 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff005d4-e21b-41c2-b860-a6e4e98df109 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.098251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff005d4-e21b-41c2-b860-a6e4e98df109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.139738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-fff005d4-e21b-41c2-b860-a6e4e98df109 None None] MFA Rules not processed for user `1f9f5086c41d407a8177298ec54126a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:28.141032 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-fff005d4-e21b-41c2-b860-a6e4e98df109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:28.216848 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1327/2658] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:28 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:28.232211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39465361-f739-4ad0-9b1d-37f0c8d092f6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:28.232211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-39465361-f739-4ad0-9b1d-37f0c8d092f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:28.291709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:28.307637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39465361-f739-4ad0-9b1d-37f0c8d092f6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:28.311651 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38971a-265a-4a93-9ca2-172873511f71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:28.312299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38971a-265a-4a93-9ca2-172873511f71 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.312577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38971a-265a-4a93-9ca2-172873511f71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.312847 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38971a-265a-4a93-9ca2-172873511f71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.323614 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38971a-265a-4a93-9ca2-172873511f71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:28.324960 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38971a-265a-4a93-9ca2-172873511f71 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:28.335936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1332/2659] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:28 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:28.370742 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff005d4-e21b-41c2-b860-a6e4e98df109 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:28.391350 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:28.440603 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff005d4-e21b-41c2-b860-a6e4e98df109 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:28.445692 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34dcd00-03d2-4b70-9a28-a17ee4e00637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:28.446391 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dcd00-03d2-4b70-9a28-a17ee4e00637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.446676 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dcd00-03d2-4b70-9a28-a17ee4e00637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.446949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dcd00-03d2-4b70-9a28-a17ee4e00637 admin admin] PATH_INFO: `/v3/users/41f1307f33a1461eb165b157845fa3f7` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.448182 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34dcd00-03d2-4b70-9a28-a17ee4e00637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41f1307f33a1461eb165b157845fa3f7)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:28.450134 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34dcd00-03d2-4b70-9a28-a17ee4e00637 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:28.504029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a34dcd00-03d2-4b70-9a28-a17ee4e00637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41f1307f33a1461eb165b157845fa3f7'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:28.504600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a34dcd00-03d2-4b70-9a28-a17ee4e00637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41f1307f33a1461eb165b157845fa3f7'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:28.505191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a34dcd00-03d2-4b70-9a28-a17ee4e00637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41f1307f33a1461eb165b157845fa3f7'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:28.514210 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a34dcd00-03d2-4b70-9a28-a17ee4e00637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41f1307f33a1461eb165b157845fa3f7'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:28.531544 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a34dcd00-03d2-4b70-9a28-a17ee4e00637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41f1307f33a1461eb165b157845fa3f7'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:28.545057 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1328/2660] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:28 2022] DELETE /identity/v3/users/41f1307f33a1461eb165b157845fa3f7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:28.550153 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfc517-dc48-4b93-bc5d-22fa1e1d055e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.550435 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfc517-dc48-4b93-bc5d-22fa1e1d055e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.550699 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfc517-dc48-4b93-bc5d-22fa1e1d055e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.570913 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-3fdfc517-dc48-4b93-bc5d-22fa1e1d055e None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:28.572135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3fdfc517-dc48-4b93-bc5d-22fa1e1d055e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:28.635431 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1333/2661] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:28.651551 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34dcd00-03d2-4b70-9a28-a17ee4e00637 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:28.653245 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a34dcd00-03d2-4b70-9a28-a17ee4e00637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:28.722652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34dcd00-03d2-4b70-9a28-a17ee4e00637 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:28.727063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2bd1ef-559d-43df-a7f2-b636338b4259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:28.727658 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bd1ef-559d-43df-a7f2-b636338b4259 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.727931 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bd1ef-559d-43df-a7f2-b636338b4259 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.728228 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bd1ef-559d-43df-a7f2-b636338b4259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.738240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2bd1ef-559d-43df-a7f2-b636338b4259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:28.739656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2bd1ef-559d-43df-a7f2-b636338b4259 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:28.749813 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1329/2662] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:29.035806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfc517-dc48-4b93-bc5d-22fa1e1d055e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:29.038197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3fdfc517-dc48-4b93-bc5d-22fa1e1d055e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:29.053502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfc517-dc48-4b93-bc5d-22fa1e1d055e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:29.060298 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba2906-855b-4adc-b614-fb72dfdb49e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:29.061013 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba2906-855b-4adc-b614-fb72dfdb49e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:29.061261 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba2906-855b-4adc-b614-fb72dfdb49e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:29.061565 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba2906-855b-4adc-b614-fb72dfdb49e9 admin admin] PATH_INFO: `/v3/projects/260581f7676c4a24bb054fb3cf3228b5` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:29.062813 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba2906-855b-4adc-b614-fb72dfdb49e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=260581f7676c4a24bb054fb3cf3228b5)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:29.064806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba2906-855b-4adc-b614-fb72dfdb49e9 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:29.116292 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-88ba2906-855b-4adc-b614-fb72dfdb49e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '260581f7676c4a24bb054fb3cf3228b5', 'request_id': 'req-88ba2906-855b-4adc-b614-fb72dfdb49e9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.125833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-88ba2906-855b-4adc-b614-fb72dfdb49e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '260581f7676c4a24bb054fb3cf3228b5', 'request_id': 'req-88ba2906-855b-4adc-b614-fb72dfdb49e9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.131667 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-88ba2906-855b-4adc-b614-fb72dfdb49e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '260581f7676c4a24bb054fb3cf3228b5', 'request_id': 'req-88ba2906-855b-4adc-b614-fb72dfdb49e9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.143414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-88ba2906-855b-4adc-b614-fb72dfdb49e9 admin admin] The token cache is being invalidate because project 260581f7676c4a24bb054fb3cf3228b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:29.143704 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-88ba2906-855b-4adc-b614-fb72dfdb49e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.144540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-88ba2906-855b-4adc-b614-fb72dfdb49e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.146038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1334/2663] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:29 2022] DELETE /identity/v3/projects/260581f7676c4a24bb054fb3cf3228b5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:29.150245 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2bd1ef-559d-43df-a7f2-b636338b4259 service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:29.160327 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2bd1ef-559d-43df-a7f2-b636338b4259 service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:29.201666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:29.223057 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a20987-52a1-42d9-b568-b2b72e54be78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:29.223690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a20987-52a1-42d9-b568-b2b72e54be78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:29.223964 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a20987-52a1-42d9-b568-b2b72e54be78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:29.224265 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a20987-52a1-42d9-b568-b2b72e54be78 admin admin] PATH_INFO: `/v3/users/1f9f5086c41d407a8177298ec54126a4` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:29.225399 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a20987-52a1-42d9-b568-b2b72e54be78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f9f5086c41d407a8177298ec54126a4)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:29.227331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a20987-52a1-42d9-b568-b2b72e54be78 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:29.279434 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a7a20987-52a1-42d9-b568-b2b72e54be78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f9f5086c41d407a8177298ec54126a4'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.280451 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a7a20987-52a1-42d9-b568-b2b72e54be78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f9f5086c41d407a8177298ec54126a4'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.281131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a7a20987-52a1-42d9-b568-b2b72e54be78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f9f5086c41d407a8177298ec54126a4'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.286360 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba2906-855b-4adc-b614-fb72dfdb49e9 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:29.287990 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-88ba2906-855b-4adc-b614-fb72dfdb49e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:29.290671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a7a20987-52a1-42d9-b568-b2b72e54be78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f9f5086c41d407a8177298ec54126a4'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.301685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba2906-855b-4adc-b614-fb72dfdb49e9 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:29.306391 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a7a20987-52a1-42d9-b568-b2b72e54be78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f9f5086c41d407a8177298ec54126a4'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.306692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d18cc-53e3-4fdb-9da7-2f248f770ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:29.307341 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d18cc-53e3-4fdb-9da7-2f248f770ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:29.307616 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d18cc-53e3-4fdb-9da7-2f248f770ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:29.307989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d18cc-53e3-4fdb-9da7-2f248f770ee6 admin admin] PATH_INFO: `/v3/users/6630cf0f761b426c88b6d95e807c7b3c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:29.309077 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d18cc-53e3-4fdb-9da7-2f248f770ee6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6630cf0f761b426c88b6d95e807c7b3c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:29.311004 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d18cc-53e3-4fdb-9da7-2f248f770ee6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:29.320918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1330/2664] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:29 2022] DELETE /identity/v3/users/1f9f5086c41d407a8177298ec54126a4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:29.367844 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-991d18cc-53e3-4fdb-9da7-2f248f770ee6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6630cf0f761b426c88b6d95e807c7b3c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.376653 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-991d18cc-53e3-4fdb-9da7-2f248f770ee6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6630cf0f761b426c88b6d95e807c7b3c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.382422 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:29.385536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-991d18cc-53e3-4fdb-9da7-2f248f770ee6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6630cf0f761b426c88b6d95e807c7b3c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.386671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-991d18cc-53e3-4fdb-9da7-2f248f770ee6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6630cf0f761b426c88b6d95e807c7b3c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.402611 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-991d18cc-53e3-4fdb-9da7-2f248f770ee6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6630cf0f761b426c88b6d95e807c7b3c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.412711 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1335/2665] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:29 2022] DELETE /identity/v3/users/6630cf0f761b426c88b6d95e807c7b3c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:29.414737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69d7de-c068-4e20-8dce-575b3d327c89 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:29.415024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69d7de-c068-4e20-8dce-575b3d327c89 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:29.415303 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69d7de-c068-4e20-8dce-575b3d327c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:29.435527 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-cd69d7de-c068-4e20-8dce-575b3d327c89 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:29.436983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-cd69d7de-c068-4e20-8dce-575b3d327c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:29.482117 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d18cc-53e3-4fdb-9da7-2f248f770ee6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:29.484968 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-991d18cc-53e3-4fdb-9da7-2f248f770ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:29.512665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1331/2666] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:29.523031 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69d7de-c068-4e20-8dce-575b3d327c89 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:29.525017 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-cd69d7de-c068-4e20-8dce-575b3d327c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:29.558527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d18cc-53e3-4fdb-9da7-2f248f770ee6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:29.562995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8024a-3017-4070-a98e-bf5fb8d0a4f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:29.563617 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8024a-3017-4070-a98e-bf5fb8d0a4f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:29.563892 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8024a-3017-4070-a98e-bf5fb8d0a4f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:29.564189 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8024a-3017-4070-a98e-bf5fb8d0a4f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:29.566521 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8ca8024a-3017-4070-a98e-bf5fb8d0a4f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:29.587104 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69d7de-c068-4e20-8dce-575b3d327c89 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:29.591800 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471ee77-c213-456d-b1a1-2e108da0c3b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:29.592195 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471ee77-c213-456d-b1a1-2e108da0c3b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:29.592603 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471ee77-c213-456d-b1a1-2e108da0c3b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:29.593115 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471ee77-c213-456d-b1a1-2e108da0c3b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:29.604137 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471ee77-c213-456d-b1a1-2e108da0c3b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:29.605653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471ee77-c213-456d-b1a1-2e108da0c3b0 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:29.617595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1332/2667] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:56:29.645137 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471ee77-c213-456d-b1a1-2e108da0c3b0 service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:29.646499 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d471ee77-c213-456d-b1a1-2e108da0c3b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:29.656953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:29.660929 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8024a-3017-4070-a98e-bf5fb8d0a4f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:29.662409 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8024a-3017-4070-a98e-bf5fb8d0a4f4 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:29.670222 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471ee77-c213-456d-b1a1-2e108da0c3b0 service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:29.678651 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb67b21-2f7b-45bc-8b06-435f0d835a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:29.680377 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb67b21-2f7b-45bc-8b06-435f0d835a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:29.680795 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1336/2668] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:56:29.681540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb67b21-2f7b-45bc-8b06-435f0d835a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:29.681875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb67b21-2f7b-45bc-8b06-435f0d835a57 admin admin] PATH_INFO: `/v3/projects/8e8fcfa98e904cb0bceb1a360fb631b1` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:29.684573 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb67b21-2f7b-45bc-8b06-435f0d835a57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e8fcfa98e904cb0bceb1a360fb631b1)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:29.687234 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb67b21-2f7b-45bc-8b06-435f0d835a57 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:29.746082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5bb67b21-2f7b-45bc-8b06-435f0d835a57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e8fcfa98e904cb0bceb1a360fb631b1', 'request_id': 'req-5bb67b21-2f7b-45bc-8b06-435f0d835a57'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.751900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5bb67b21-2f7b-45bc-8b06-435f0d835a57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e8fcfa98e904cb0bceb1a360fb631b1', 'request_id': 'req-5bb67b21-2f7b-45bc-8b06-435f0d835a57'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.758486 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5bb67b21-2f7b-45bc-8b06-435f0d835a57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e8fcfa98e904cb0bceb1a360fb631b1', 'request_id': 'req-5bb67b21-2f7b-45bc-8b06-435f0d835a57'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.773564 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5bb67b21-2f7b-45bc-8b06-435f0d835a57 admin admin] The token cache is being invalidate because project 8e8fcfa98e904cb0bceb1a360fb631b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:29.773880 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5bb67b21-2f7b-45bc-8b06-435f0d835a57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.774482 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5bb67b21-2f7b-45bc-8b06-435f0d835a57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:29.776233 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1333/2669] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:29 2022] DELETE /identity/v3/projects/8e8fcfa98e904cb0bceb1a360fb631b1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:56:29.782296 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844b9d8-13b2-4b81-8dca-eb350678bba1 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:29.782575 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844b9d8-13b2-4b81-8dca-eb350678bba1 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:29.782842 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844b9d8-13b2-4b81-8dca-eb350678bba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:29.803538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9844b9d8-13b2-4b81-8dca-eb350678bba1 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:29.804826 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9844b9d8-13b2-4b81-8dca-eb350678bba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:29.871929 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1337/2670] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:56:29.880120 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0bd5ef-1de9-4411-a64b-a3aed0da2904 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:29.880629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0bd5ef-1de9-4411-a64b-a3aed0da2904 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:29.880909 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0bd5ef-1de9-4411-a64b-a3aed0da2904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:29.903456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-af0bd5ef-1de9-4411-a64b-a3aed0da2904 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:29.904875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-af0bd5ef-1de9-4411-a64b-a3aed0da2904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:29.914200 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1334/2671] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:56:29.919221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844b9d8-13b2-4b81-8dca-eb350678bba1 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:29.932112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0bd5ef-1de9-4411-a64b-a3aed0da2904 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:29.935829 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844b9d8-13b2-4b81-8dca-eb350678bba1 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:29.937177 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-af0bd5ef-1de9-4411-a64b-a3aed0da2904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:29.945287 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9323f33-4e74-48a7-ba3f-5d9542b44e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:29.948647 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9323f33-4e74-48a7-ba3f-5d9542b44e4a admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:29.948923 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9323f33-4e74-48a7-ba3f-5d9542b44e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:29.949197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9323f33-4e74-48a7-ba3f-5d9542b44e4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:29.949867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9323f33-4e74-48a7-ba3f-5d9542b44e4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:29.950168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9323f33-4e74-48a7-ba3f-5d9542b44e4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:29.951841 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9323f33-4e74-48a7-ba3f-5d9542b44e4a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:29.957164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0bd5ef-1de9-4411-a64b-a3aed0da2904 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:29.968471 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:29.969366 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:29.969366 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:29.969618 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7 admin admin] PATH_INFO: `/v3/projects/a663360806e041e0a6adfb97b92e3b33` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:29.970850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a663360806e041e0a6adfb97b92e3b33)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:29.972801 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:29.976172 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1338/2672] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:29.979037 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9323f33-4e74-48a7-ba3f-5d9542b44e4a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:29.995093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9323f33-4e74-48a7-ba3f-5d9542b44e4a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:30.002857 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0eadf-f8ab-4b0f-8b74-01dbf0d6257a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:30.003833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0eadf-f8ab-4b0f-8b74-01dbf0d6257a admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:30.004836 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0eadf-f8ab-4b0f-8b74-01dbf0d6257a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:30.005401 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0eadf-f8ab-4b0f-8b74-01dbf0d6257a admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:30.006716 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0eadf-f8ab-4b0f-8b74-01dbf0d6257a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:30.010179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0eadf-f8ab-4b0f-8b74-01dbf0d6257a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:30.024812 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a663360806e041e0a6adfb97b92e3b33', 'request_id': 'req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:30.029837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:30.033267 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a663360806e041e0a6adfb97b92e3b33', 'request_id': 'req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:30.041310 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a663360806e041e0a6adfb97b92e3b33', 'request_id': 'req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:30.048104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1339/2673] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:29 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:56:30.054383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0eadf-f8ab-4b0f-8b74-01dbf0d6257a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:30.060896 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7 admin admin] The token cache is being invalidate because project a663360806e041e0a6adfb97b92e3b33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:30.061420 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:30.062281 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:30.064780 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1335/2674] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:29 2022] DELETE /identity/v3/projects/a663360806e041e0a6adfb97b92e3b33 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:56:30.141873 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0eadf-f8ab-4b0f-8b74-01dbf0d6257a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:30.148408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85304a1-e66c-4bfb-88f6-a18932be3c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:30.149270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85304a1-e66c-4bfb-88f6-a18932be3c4d admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:30.149615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85304a1-e66c-4bfb-88f6-a18932be3c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:30.149952 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85304a1-e66c-4bfb-88f6-a18932be3c4d admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:30.150698 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85304a1-e66c-4bfb-88f6-a18932be3c4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:30.152840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85304a1-e66c-4bfb-88f6-a18932be3c4d admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:30.178066 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1340/2675] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:30 2022] POST /identity/v3/users => generated 290 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:56:30.182772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:30.184310 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:30.273037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd817b7b-1970-4f8b-a1a6-0f705b6bafa7 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:30.281287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd329ab-66b9-4d25-8b8d-ad8ac7044a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:30.282417 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd329ab-66b9-4d25-8b8d-ad8ac7044a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:30.282572 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd329ab-66b9-4d25-8b8d-ad8ac7044a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:30.282947 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd329ab-66b9-4d25-8b8d-ad8ac7044a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:30.283734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd329ab-66b9-4d25-8b8d-ad8ac7044a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:30.286047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd329ab-66b9-4d25-8b8d-ad8ac7044a97 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:30.289952 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:30.306186 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1336/2676] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:30.307755 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85304a1-e66c-4bfb-88f6-a18932be3c4d admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:30.327199 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85304a1-e66c-4bfb-88f6-a18932be3c4d admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:30.334103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d491684-748b-49e4-a489-f1663bca01c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:30.335192 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d491684-748b-49e4-a489-f1663bca01c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:30.336070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d491684-748b-49e4-a489-f1663bca01c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:30.336736 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d491684-748b-49e4-a489-f1663bca01c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:30.338709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d491684-748b-49e4-a489-f1663bca01c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:30.340598 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d491684-748b-49e4-a489-f1663bca01c2 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:30.354047 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1341/2677] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:30.358245 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd329ab-66b9-4d25-8b8d-ad8ac7044a97 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:30.370369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd329ab-66b9-4d25-8b8d-ad8ac7044a97 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:30.375176 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cebd56-cef8-421e-aa01-66c9c720ceb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:30.375837 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cebd56-cef8-421e-aa01-66c9c720ceb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:30.376257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cebd56-cef8-421e-aa01-66c9c720ceb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:30.376472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cebd56-cef8-421e-aa01-66c9c720ceb6 admin admin] PATH_INFO: `/v3/projects/4abe96eff6ca4e8ba74b63aee50e1bc9/users/ead1129d77a0491e84a3816472f36b03/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:30.392820 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cebd56-cef8-421e-aa01-66c9c720ceb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4abe96eff6ca4e8ba74b63aee50e1bc9, user_id=ead1129d77a0491e84a3816472f36b03, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:30.395368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cebd56-cef8-421e-aa01-66c9c720ceb6 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:30.410615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1337/2678] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:30 2022] PUT /identity/v3/projects/4abe96eff6ca4e8ba74b63aee50e1bc9/users/ead1129d77a0491e84a3816472f36b03/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:30.422269 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1cd69-ef1d-4ccc-a7e4-ac34388c9007 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:30.422572 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1cd69-ef1d-4ccc-a7e4-ac34388c9007 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:30.422793 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1cd69-ef1d-4ccc-a7e4-ac34388c9007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:30.454417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-86c1cd69-ef1d-4ccc-a7e4-ac34388c9007 None None] MFA Rules not processed for user `ead1129d77a0491e84a3816472f36b03`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:30.455843 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-86c1cd69-ef1d-4ccc-a7e4-ac34388c9007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:30.528570 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1342/2679] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:30.532968 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cebd56-cef8-421e-aa01-66c9c720ceb6 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:30.599123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cebd56-cef8-421e-aa01-66c9c720ceb6 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:30.604411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e786634-d744-4315-b0ea-2dac345488ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:30.605032 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e786634-d744-4315-b0ea-2dac345488ba admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:30.605306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e786634-d744-4315-b0ea-2dac345488ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:30.605576 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e786634-d744-4315-b0ea-2dac345488ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:30.606223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e786634-d744-4315-b0ea-2dac345488ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:30.608409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e786634-d744-4315-b0ea-2dac345488ba admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:30.645426 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1338/2680] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:30 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:30.649730 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1cd69-ef1d-4ccc-a7e4-ac34388c9007 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:30.661117 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1cd69-ef1d-4ccc-a7e4-ac34388c9007 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:30.732641 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b145799-a375-4c8a-9b77-a9d2145b494b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:30.733060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b145799-a375-4c8a-9b77-a9d2145b494b admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:30.733349 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b145799-a375-4c8a-9b77-a9d2145b494b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:30.733611 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b145799-a375-4c8a-9b77-a9d2145b494b admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:30.734394 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b145799-a375-4c8a-9b77-a9d2145b494b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:30.736380 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b145799-a375-4c8a-9b77-a9d2145b494b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:30.745897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:30.760588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1343/2681] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:30 2022] POST /identity/v3/users => generated 296 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:30.764684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e786634-d744-4315-b0ea-2dac345488ba admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:30.776508 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e786634-d744-4315-b0ea-2dac345488ba admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:30.781422 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc6f9e-099f-435e-b989-eb2f6de9a449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:30.782035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc6f9e-099f-435e-b989-eb2f6de9a449 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:30.782312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc6f9e-099f-435e-b989-eb2f6de9a449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:30.782582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc6f9e-099f-435e-b989-eb2f6de9a449 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:30.783232 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc6f9e-099f-435e-b989-eb2f6de9a449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:30.784858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc6f9e-099f-435e-b989-eb2f6de9a449 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:30.796396 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1339/2682] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:30.801187 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b145799-a375-4c8a-9b77-a9d2145b494b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:30.811607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b145799-a375-4c8a-9b77-a9d2145b494b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:30.816393 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97406fc-3a8c-4224-8948-edb1bfc0a7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:30.817059 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97406fc-3a8c-4224-8948-edb1bfc0a7a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:30.817331 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97406fc-3a8c-4224-8948-edb1bfc0a7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:30.817599 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97406fc-3a8c-4224-8948-edb1bfc0a7a3 admin admin] PATH_INFO: `/v3/projects/6d946ca5bcf8436ca92d3ef1acc3af04/users/bfb3f8dbcb9c44db82f666404eb9cd13/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:30.833352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97406fc-3a8c-4224-8948-edb1bfc0a7a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d946ca5bcf8436ca92d3ef1acc3af04, user_id=bfb3f8dbcb9c44db82f666404eb9cd13, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:30.836094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97406fc-3a8c-4224-8948-edb1bfc0a7a3 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:30.851761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1344/2683] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:30 2022] PUT /identity/v3/projects/6d946ca5bcf8436ca92d3ef1acc3af04/users/bfb3f8dbcb9c44db82f666404eb9cd13/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:56:30.855656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc6f9e-099f-435e-b989-eb2f6de9a449 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:30.865721 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc6f9e-099f-435e-b989-eb2f6de9a449 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:30.927035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61371552-b4c7-4499-ba33-97760aad2628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:30.927652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61371552-b4c7-4499-ba33-97760aad2628 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:30.927929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61371552-b4c7-4499-ba33-97760aad2628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:30.928244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61371552-b4c7-4499-ba33-97760aad2628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:30.929036 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61371552-b4c7-4499-ba33-97760aad2628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:30.930709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61371552-b4c7-4499-ba33-97760aad2628 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:30.941045 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1340/2684] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:30.945686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97406fc-3a8c-4224-8948-edb1bfc0a7a3 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:30.958143 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97406fc-3a8c-4224-8948-edb1bfc0a7a3 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:30.964291 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb51fbe-1b48-40f5-b9e4-d8afc0af5b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:30.965046 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb51fbe-1b48-40f5-b9e4-d8afc0af5b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:30.965397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb51fbe-1b48-40f5-b9e4-d8afc0af5b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:30.965668 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb51fbe-1b48-40f5-b9e4-d8afc0af5b6e admin admin] PATH_INFO: `/v3/projects/6d946ca5bcf8436ca92d3ef1acc3af04/users/bfb3f8dbcb9c44db82f666404eb9cd13/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:30.967823 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb51fbe-1b48-40f5-b9e4-d8afc0af5b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d946ca5bcf8436ca92d3ef1acc3af04, user_id=bfb3f8dbcb9c44db82f666404eb9cd13, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:30.970736 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb51fbe-1b48-40f5-b9e4-d8afc0af5b6e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:30.985133 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1345/2685] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:30 2022] PUT /identity/v3/projects/6d946ca5bcf8436ca92d3ef1acc3af04/users/bfb3f8dbcb9c44db82f666404eb9cd13/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:30.995331 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a2b1b-d7ed-41a3-a836-677e35e10b60 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:30.995621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a2b1b-d7ed-41a3-a836-677e35e10b60 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:30.995879 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a2b1b-d7ed-41a3-a836-677e35e10b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:31.028365 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-9a6a2b1b-d7ed-41a3-a836-677e35e10b60 None None] MFA Rules not processed for user `bfb3f8dbcb9c44db82f666404eb9cd13`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:31.030024 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-9a6a2b1b-d7ed-41a3-a836-677e35e10b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:31.065601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:31.106549 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1341/2686] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:31.118035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb51fbe-1b48-40f5-b9e4-d8afc0af5b6e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:31.119821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0bb51fbe-1b48-40f5-b9e4-d8afc0af5b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:31.179932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb51fbe-1b48-40f5-b9e4-d8afc0af5b6e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:31.183703 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c33e1-6611-4d63-965b-735d0f8439f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:31.184423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c33e1-6611-4d63-965b-735d0f8439f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:31.184766 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c33e1-6611-4d63-965b-735d0f8439f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:31.185085 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c33e1-6611-4d63-965b-735d0f8439f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:31.194960 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c33e1-6611-4d63-965b-735d0f8439f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:31.196661 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c33e1-6611-4d63-965b-735d0f8439f9 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:31.206694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1346/2687] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:31.615463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a2b1b-d7ed-41a3-a836-677e35e10b60 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:31.698562 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a2b1b-d7ed-41a3-a836-677e35e10b60 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:31.703399 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c71ad7-245a-41a9-875d-73d541da1a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:31.704079 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c71ad7-245a-41a9-875d-73d541da1a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:31.704358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c71ad7-245a-41a9-875d-73d541da1a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:31.704633 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c71ad7-245a-41a9-875d-73d541da1a49 admin admin] PATH_INFO: `/v3/users/ead1129d77a0491e84a3816472f36b03` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:31.705802 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c71ad7-245a-41a9-875d-73d541da1a49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ead1129d77a0491e84a3816472f36b03)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:31.707790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c71ad7-245a-41a9-875d-73d541da1a49 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:31.763551 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-79c71ad7-245a-41a9-875d-73d541da1a49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ead1129d77a0491e84a3816472f36b03'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:31.764304 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-79c71ad7-245a-41a9-875d-73d541da1a49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ead1129d77a0491e84a3816472f36b03'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:31.765088 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-79c71ad7-245a-41a9-875d-73d541da1a49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ead1129d77a0491e84a3816472f36b03'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:31.777429 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-79c71ad7-245a-41a9-875d-73d541da1a49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ead1129d77a0491e84a3816472f36b03'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:31.792625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-79c71ad7-245a-41a9-875d-73d541da1a49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ead1129d77a0491e84a3816472f36b03'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:31.806823 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1342/2688] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:31 2022] DELETE /identity/v3/users/ead1129d77a0491e84a3816472f36b03 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:31.811295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f61cd-b57c-4a97-904b-0447bf132384 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:31.811789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f61cd-b57c-4a97-904b-0447bf132384 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:31.812253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f61cd-b57c-4a97-904b-0447bf132384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:31.832443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f18f61cd-b57c-4a97-904b-0447bf132384 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:31.833674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f18f61cd-b57c-4a97-904b-0447bf132384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:31.910539 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1347/2689] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:31.920251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c71ad7-245a-41a9-875d-73d541da1a49 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:31.922091 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-79c71ad7-245a-41a9-875d-73d541da1a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:31.990206 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:31.994902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c71ad7-245a-41a9-875d-73d541da1a49 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:31.999250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096e77d-5ef1-4975-a9d1-bd76f4ae81fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:32.000848 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096e77d-5ef1-4975-a9d1-bd76f4ae81fa service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:32.000848 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096e77d-5ef1-4975-a9d1-bd76f4ae81fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:32.001096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096e77d-5ef1-4975-a9d1-bd76f4ae81fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:32.013371 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096e77d-5ef1-4975-a9d1-bd76f4ae81fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:32.015047 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096e77d-5ef1-4975-a9d1-bd76f4ae81fa service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:32.028255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1343/2690] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:56:32.325840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f61cd-b57c-4a97-904b-0447bf132384 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:32.328843 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f18f61cd-b57c-4a97-904b-0447bf132384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:32.358317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f61cd-b57c-4a97-904b-0447bf132384 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:32.363587 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:32.364362 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:32.364796 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:32.365034 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36 admin admin] PATH_INFO: `/v3/projects/4abe96eff6ca4e8ba74b63aee50e1bc9` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:32.366401 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4abe96eff6ca4e8ba74b63aee50e1bc9)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:32.368712 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:32.422311 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4abe96eff6ca4e8ba74b63aee50e1bc9', 'request_id': 'req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:32.425637 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:32.432725 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4abe96eff6ca4e8ba74b63aee50e1bc9', 'request_id': 'req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:32.439534 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4abe96eff6ca4e8ba74b63aee50e1bc9', 'request_id': 'req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:32.452440 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36 admin admin] The token cache is being invalidate because project 4abe96eff6ca4e8ba74b63aee50e1bc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:32.452630 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:32.453282 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:32.454784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1348/2691] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:32 2022] DELETE /identity/v3/projects/4abe96eff6ca4e8ba74b63aee50e1bc9 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:32.458856 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096e77d-5ef1-4975-a9d1-bd76f4ae81fa service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:32.469404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096e77d-5ef1-4975-a9d1-bd76f4ae81fa service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:32.534359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b33357-ed95-468a-bbcc-da56fae8d66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:32.535258 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b33357-ed95-468a-bbcc-da56fae8d66b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:32.535718 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b33357-ed95-468a-bbcc-da56fae8d66b admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:32.536013 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b33357-ed95-468a-bbcc-da56fae8d66b admin admin] PATH_INFO: `/v3/users/bfb3f8dbcb9c44db82f666404eb9cd13` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:32.537247 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b33357-ed95-468a-bbcc-da56fae8d66b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfb3f8dbcb9c44db82f666404eb9cd13)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:32.539267 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b33357-ed95-468a-bbcc-da56fae8d66b admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:32.604998 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f2b33357-ed95-468a-bbcc-da56fae8d66b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfb3f8dbcb9c44db82f666404eb9cd13'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:32.605661 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f2b33357-ed95-468a-bbcc-da56fae8d66b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfb3f8dbcb9c44db82f666404eb9cd13'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:32.606266 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f2b33357-ed95-468a-bbcc-da56fae8d66b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfb3f8dbcb9c44db82f666404eb9cd13'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:32.615291 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f2b33357-ed95-468a-bbcc-da56fae8d66b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfb3f8dbcb9c44db82f666404eb9cd13'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:32.630764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-f2b33357-ed95-468a-bbcc-da56fae8d66b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfb3f8dbcb9c44db82f666404eb9cd13'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:32.643916 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1344/2692] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:32 2022] DELETE /identity/v3/users/bfb3f8dbcb9c44db82f666404eb9cd13 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:32.932969 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:32.934418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:32.952959 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c0eac-7d35-44b3-a0b6-3d4dd915bc36 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:33.013981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8a590-a580-4e0c-89db-3c6bf7bf1166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:33.014523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8a590-a580-4e0c-89db-3c6bf7bf1166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:33.014799 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8a590-a580-4e0c-89db-3c6bf7bf1166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:33.015061 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8a590-a580-4e0c-89db-3c6bf7bf1166 admin admin] PATH_INFO: `/v3/projects/6d946ca5bcf8436ca92d3ef1acc3af04` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:33.016252 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8a590-a580-4e0c-89db-3c6bf7bf1166 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d946ca5bcf8436ca92d3ef1acc3af04)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:33.018169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8a590-a580-4e0c-89db-3c6bf7bf1166 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:33.069698 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-27e8a590-a580-4e0c-89db-3c6bf7bf1166 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d946ca5bcf8436ca92d3ef1acc3af04', 'request_id': 'req-27e8a590-a580-4e0c-89db-3c6bf7bf1166'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:33.080029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-27e8a590-a580-4e0c-89db-3c6bf7bf1166 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d946ca5bcf8436ca92d3ef1acc3af04', 'request_id': 'req-27e8a590-a580-4e0c-89db-3c6bf7bf1166'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:33.086977 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-27e8a590-a580-4e0c-89db-3c6bf7bf1166 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d946ca5bcf8436ca92d3ef1acc3af04', 'request_id': 'req-27e8a590-a580-4e0c-89db-3c6bf7bf1166'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:33.100415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-27e8a590-a580-4e0c-89db-3c6bf7bf1166 admin admin] The token cache is being invalidate because project 6d946ca5bcf8436ca92d3ef1acc3af04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:33.100712 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-27e8a590-a580-4e0c-89db-3c6bf7bf1166 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:33.102092 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-27e8a590-a580-4e0c-89db-3c6bf7bf1166 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:33.103698 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1349/2693] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:32 2022] DELETE /identity/v3/projects/6d946ca5bcf8436ca92d3ef1acc3af04 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:33.108899 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c669d1-39bc-4f89-a018-1b54f296094d None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:33.109179 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c669d1-39bc-4f89-a018-1b54f296094d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:33.109449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c669d1-39bc-4f89-a018-1b54f296094d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:33.130940 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-61c669d1-39bc-4f89-a018-1b54f296094d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:33.132308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-61c669d1-39bc-4f89-a018-1b54f296094d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:33.199058 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1345/2694] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:56:33.208259 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47c785-1a5f-4205-a853-18e025dd031e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:33.208539 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47c785-1a5f-4205-a853-18e025dd031e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:33.208791 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47c785-1a5f-4205-a853-18e025dd031e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:33.230418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ce47c785-1a5f-4205-a853-18e025dd031e None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:33.231645 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ce47c785-1a5f-4205-a853-18e025dd031e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:33.240854 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1350/2695] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:33.245427 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c669d1-39bc-4f89-a018-1b54f296094d None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:33.256896 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c669d1-39bc-4f89-a018-1b54f296094d None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:33.264606 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5588f6-8531-4c72-97ad-a4023fd9d9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:33.265280 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5588f6-8531-4c72-97ad-a4023fd9d9af admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:33.265658 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5588f6-8531-4c72-97ad-a4023fd9d9af admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:33.266027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5588f6-8531-4c72-97ad-a4023fd9d9af admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:33.266820 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5588f6-8531-4c72-97ad-a4023fd9d9af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:33.267310 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5588f6-8531-4c72-97ad-a4023fd9d9af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:33.269312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5588f6-8531-4c72-97ad-a4023fd9d9af admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:33.277236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:33.284105 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1346/2696] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:33.289397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47c785-1a5f-4205-a853-18e025dd031e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:33.298806 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47c785-1a5f-4205-a853-18e025dd031e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:33.304511 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3025f5-ed26-4d66-9c2f-baa27b2d9cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:33.305149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3025f5-ed26-4d66-9c2f-baa27b2d9cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:33.305423 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3025f5-ed26-4d66-9c2f-baa27b2d9cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:33.305696 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3025f5-ed26-4d66-9c2f-baa27b2d9cd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:33.306326 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3025f5-ed26-4d66-9c2f-baa27b2d9cd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:33.308300 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3025f5-ed26-4d66-9c2f-baa27b2d9cd2 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:33.341170 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1351/2697] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:33 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:33.345477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5588f6-8531-4c72-97ad-a4023fd9d9af admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:33.356637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5588f6-8531-4c72-97ad-a4023fd9d9af admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:33.420405 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397030c0-1b55-49f1-90de-3fe383f46880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:33.421061 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397030c0-1b55-49f1-90de-3fe383f46880 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:33.421339 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397030c0-1b55-49f1-90de-3fe383f46880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:33.421570 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397030c0-1b55-49f1-90de-3fe383f46880 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:33.422211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397030c0-1b55-49f1-90de-3fe383f46880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:33.424186 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397030c0-1b55-49f1-90de-3fe383f46880 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:33.448464 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1347/2698] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:33 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:33.452920 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3025f5-ed26-4d66-9c2f-baa27b2d9cd2 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:33.463893 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3025f5-ed26-4d66-9c2f-baa27b2d9cd2 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:33.469495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81222d5b-32ae-4555-99d7-b47b7ced8b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:33.470128 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81222d5b-32ae-4555-99d7-b47b7ced8b3a admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:33.470465 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81222d5b-32ae-4555-99d7-b47b7ced8b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:33.470739 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81222d5b-32ae-4555-99d7-b47b7ced8b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:33.471851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81222d5b-32ae-4555-99d7-b47b7ced8b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:33.473607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81222d5b-32ae-4555-99d7-b47b7ced8b3a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:33.484904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1352/2699] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:33.490668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397030c0-1b55-49f1-90de-3fe383f46880 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:33.502606 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397030c0-1b55-49f1-90de-3fe383f46880 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:33.508388 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5951079-8919-4910-9916-b673d880c278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:33.508986 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5951079-8919-4910-9916-b673d880c278 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:33.509261 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5951079-8919-4910-9916-b673d880c278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:33.509525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5951079-8919-4910-9916-b673d880c278 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:33.510164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5951079-8919-4910-9916-b673d880c278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:33.511733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5951079-8919-4910-9916-b673d880c278 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:33.523482 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1348/2700] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:33.528222 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81222d5b-32ae-4555-99d7-b47b7ced8b3a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:33.541949 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81222d5b-32ae-4555-99d7-b47b7ced8b3a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:33.548012 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d9372-d8fa-457c-a6c9-aa542eed7bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:33.548684 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d9372-d8fa-457c-a6c9-aa542eed7bd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:33.548958 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d9372-d8fa-457c-a6c9-aa542eed7bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:33.549226 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d9372-d8fa-457c-a6c9-aa542eed7bd7 admin admin] PATH_INFO: `/v3/projects/d82dce8fbe2846debf5ee4efc1e74bdb/users/e557fdf74add40dca1335f24242128dc/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:33.565712 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d9372-d8fa-457c-a6c9-aa542eed7bd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d82dce8fbe2846debf5ee4efc1e74bdb, user_id=e557fdf74add40dca1335f24242128dc, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:33.568438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d9372-d8fa-457c-a6c9-aa542eed7bd7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:33.583526 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1353/2701] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:33 2022] PUT /identity/v3/projects/d82dce8fbe2846debf5ee4efc1e74bdb/users/e557fdf74add40dca1335f24242128dc/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:33.588989 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad1008-6420-495c-b9f9-43bcd96f2b48 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:33.589274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad1008-6420-495c-b9f9-43bcd96f2b48 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:33.590039 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad1008-6420-495c-b9f9-43bcd96f2b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:33.609974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-0bad1008-6420-495c-b9f9-43bcd96f2b48 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:33.611189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0bad1008-6420-495c-b9f9-43bcd96f2b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:33.673809 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1349/2702] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:33.684205 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d9372-d8fa-457c-a6c9-aa542eed7bd7 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:33.686891 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ac8d9372-d8fa-457c-a6c9-aa542eed7bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:33.701093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:33.755002 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d9372-d8fa-457c-a6c9-aa542eed7bd7 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:33.759780 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed62a8-3158-4614-84dd-7ccc15a8c01f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:33.760443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed62a8-3158-4614-84dd-7ccc15a8c01f service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:33.760711 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed62a8-3158-4614-84dd-7ccc15a8c01f service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:33.760979 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed62a8-3158-4614-84dd-7ccc15a8c01f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:33.771615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed62a8-3158-4614-84dd-7ccc15a8c01f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:33.773573 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed62a8-3158-4614-84dd-7ccc15a8c01f service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:33.783679 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1354/2703] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:34.459454 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5b85-b4fa-419a-972a-2ac6bf06fd8d None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:34.459743 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5b85-b4fa-419a-972a-2ac6bf06fd8d None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:34.460087 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5b85-b4fa-419a-972a-2ac6bf06fd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:34.492310 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-80ca5b85-b4fa-419a-972a-2ac6bf06fd8d None None] MFA Rules not processed for user `e557fdf74add40dca1335f24242128dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:34.493704 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-80ca5b85-b4fa-419a-972a-2ac6bf06fd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:34.566938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1350/2704] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:34.571597 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed62a8-3158-4614-84dd-7ccc15a8c01f service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:34.582013 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed62a8-3158-4614-84dd-7ccc15a8c01f service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:34.587263 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db759a-9482-45ad-97eb-06e4572998c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:34.587877 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db759a-9482-45ad-97eb-06e4572998c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:34.588310 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db759a-9482-45ad-97eb-06e4572998c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:34.588578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db759a-9482-45ad-97eb-06e4572998c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:34.589215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db759a-9482-45ad-97eb-06e4572998c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:34.591129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db759a-9482-45ad-97eb-06e4572998c6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:34.628129 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1355/2705] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:34 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:34.633873 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca5b85-b4fa-419a-972a-2ac6bf06fd8d None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:34.647250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca5b85-b4fa-419a-972a-2ac6bf06fd8d None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:34.651935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:34.710834 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11953d-ef34-474e-b8b1-b152d91f1865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:34.711574 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11953d-ef34-474e-b8b1-b152d91f1865 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:34.711780 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11953d-ef34-474e-b8b1-b152d91f1865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:34.712077 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11953d-ef34-474e-b8b1-b152d91f1865 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:34.712723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11953d-ef34-474e-b8b1-b152d91f1865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:34.714717 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11953d-ef34-474e-b8b1-b152d91f1865 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:34.738663 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1351/2706] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:34 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:34.743488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db759a-9482-45ad-97eb-06e4572998c6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:34.754415 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db759a-9482-45ad-97eb-06e4572998c6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:34.760448 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24572348-4e00-4c05-9cec-11e41be735d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:34.761059 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24572348-4e00-4c05-9cec-11e41be735d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:34.761343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24572348-4e00-4c05-9cec-11e41be735d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:34.761615 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24572348-4e00-4c05-9cec-11e41be735d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:34.762254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24572348-4e00-4c05-9cec-11e41be735d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:34.764031 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24572348-4e00-4c05-9cec-11e41be735d9 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:34.775980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1356/2707] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:34.780880 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11953d-ef34-474e-b8b1-b152d91f1865 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:34.791607 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11953d-ef34-474e-b8b1-b152d91f1865 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:34.797307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27003e40-83b2-4522-bc0c-d103762a22f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:34.797961 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27003e40-83b2-4522-bc0c-d103762a22f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:34.798243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27003e40-83b2-4522-bc0c-d103762a22f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:34.798518 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27003e40-83b2-4522-bc0c-d103762a22f7 admin admin] PATH_INFO: `/v3/projects/233672949cdd4636b2f95a6889c66058/users/572b77ffe3114453a45d69e3a6a915c0/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:34.814437 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27003e40-83b2-4522-bc0c-d103762a22f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=233672949cdd4636b2f95a6889c66058, user_id=572b77ffe3114453a45d69e3a6a915c0, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:34.817196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27003e40-83b2-4522-bc0c-d103762a22f7 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:34.833025 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1352/2708] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:34 2022] PUT /identity/v3/projects/233672949cdd4636b2f95a6889c66058/users/572b77ffe3114453a45d69e3a6a915c0/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:34.837898 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24572348-4e00-4c05-9cec-11e41be735d9 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:34.853256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24572348-4e00-4c05-9cec-11e41be735d9 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:34.912516 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08e155-a42a-4a01-985d-1fa6698e730b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:34.913240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08e155-a42a-4a01-985d-1fa6698e730b admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:34.913514 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08e155-a42a-4a01-985d-1fa6698e730b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:34.913782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08e155-a42a-4a01-985d-1fa6698e730b admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:34.914453 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08e155-a42a-4a01-985d-1fa6698e730b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:34.916078 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08e155-a42a-4a01-985d-1fa6698e730b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:34.926320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1357/2709] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:34.930919 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27003e40-83b2-4522-bc0c-d103762a22f7 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:34.941632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27003e40-83b2-4522-bc0c-d103762a22f7 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:34.947441 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e5b92-f670-4361-9bc7-d75b16ee1556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:34.948159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e5b92-f670-4361-9bc7-d75b16ee1556 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:34.948462 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e5b92-f670-4361-9bc7-d75b16ee1556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:34.948731 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e5b92-f670-4361-9bc7-d75b16ee1556 admin admin] PATH_INFO: `/v3/projects/233672949cdd4636b2f95a6889c66058/users/572b77ffe3114453a45d69e3a6a915c0/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:34.951137 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e5b92-f670-4361-9bc7-d75b16ee1556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=233672949cdd4636b2f95a6889c66058, user_id=572b77ffe3114453a45d69e3a6a915c0, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:34.953828 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e5b92-f670-4361-9bc7-d75b16ee1556 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:34.965498 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1353/2710] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:34 2022] PUT /identity/v3/projects/233672949cdd4636b2f95a6889c66058/users/572b77ffe3114453a45d69e3a6a915c0/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:35.646803 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06331ee-bf25-466c-990e-fb3de378e817 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:35.647382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06331ee-bf25-466c-990e-fb3de378e817 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:35.647382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06331ee-bf25-466c-990e-fb3de378e817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:35.678904 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f06331ee-bf25-466c-990e-fb3de378e817 None None] MFA Rules not processed for user `572b77ffe3114453a45d69e3a6a915c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:35.680488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f06331ee-bf25-466c-990e-fb3de378e817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:35.751657 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:35.766807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1358/2711] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:35.771892 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e5b92-f670-4361-9bc7-d75b16ee1556 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:35.774315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-100e5b92-f670-4361-9bc7-d75b16ee1556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:35.837021 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e5b92-f670-4361-9bc7-d75b16ee1556 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:35.840847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d73a89-3bf7-4694-88e3-0a78ab638536 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:35.841464 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d73a89-3bf7-4694-88e3-0a78ab638536 service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:35.841737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d73a89-3bf7-4694-88e3-0a78ab638536 service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:35.842001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d73a89-3bf7-4694-88e3-0a78ab638536 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:35.897345 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d73a89-3bf7-4694-88e3-0a78ab638536 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:35.898858 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d73a89-3bf7-4694-88e3-0a78ab638536 service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:35.908567 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1354/2712] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:37.311299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06331ee-bf25-466c-990e-fb3de378e817 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:37.313074 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f06331ee-bf25-466c-990e-fb3de378e817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:37.327246 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06331ee-bf25-466c-990e-fb3de378e817 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:37.331600 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d36dec-851e-41e3-8dc6-4474b061d833 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:37.332256 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d36dec-851e-41e3-8dc6-4474b061d833 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:37.332530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d36dec-851e-41e3-8dc6-4474b061d833 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:37.332822 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d36dec-851e-41e3-8dc6-4474b061d833 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:37.343196 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d36dec-851e-41e3-8dc6-4474b061d833 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:37.344674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d36dec-851e-41e3-8dc6-4474b061d833 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:37.354141 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1359/2713] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:45.904257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:INSERT=5 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:47.350081 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=7 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:50.116616 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d73a89-3bf7-4694-88e3-0a78ab638536 service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:50.118974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-10d73a89-3bf7-4694-88e3-0a78ab638536 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:50.224566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d73a89-3bf7-4694-88e3-0a78ab638536 service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:50.229844 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd219a4b-4d5c-4543-8cd2-d2c6380485f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cdd6b1ee9f4e97beb9bb037e446c8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:50.230546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd219a4b-4d5c-4543-8cd2-d2c6380485f0 service placement] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:50.230853 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd219a4b-4d5c-4543-8cd2-d2c6380485f0 service placement] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:50.231090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd219a4b-4d5c-4543-8cd2-d2c6380485f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:50.233022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-bd219a4b-4d5c-4543-8cd2-d2c6380485f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:50.245232 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd219a4b-4d5c-4543-8cd2-d2c6380485f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:50.246806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd219a4b-4d5c-4543-8cd2-d2c6380485f0 service placement] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:50.257094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1355/2714] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:50.767050 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d36dec-851e-41e3-8dc6-4474b061d833 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:50.853151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d36dec-851e-41e3-8dc6-4474b061d833 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:50.857894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd820be-f601-4541-aa8c-4e4761c29851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:50.858517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd820be-f601-4541-aa8c-4e4761c29851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:50.858789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd820be-f601-4541-aa8c-4e4761c29851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:50.859055 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd820be-f601-4541-aa8c-4e4761c29851 admin admin] PATH_INFO: `/v3/users/e557fdf74add40dca1335f24242128dc` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:50.860222 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd820be-f601-4541-aa8c-4e4761c29851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e557fdf74add40dca1335f24242128dc)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:50.862127 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd820be-f601-4541-aa8c-4e4761c29851 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:50.917546 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ebd820be-f601-4541-aa8c-4e4761c29851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e557fdf74add40dca1335f24242128dc'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:50.925063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ebd820be-f601-4541-aa8c-4e4761c29851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e557fdf74add40dca1335f24242128dc'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:50.934304 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ebd820be-f601-4541-aa8c-4e4761c29851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e557fdf74add40dca1335f24242128dc'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:50.935033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ebd820be-f601-4541-aa8c-4e4761c29851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e557fdf74add40dca1335f24242128dc'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:50.947235 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ebd820be-f601-4541-aa8c-4e4761c29851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e557fdf74add40dca1335f24242128dc'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:50.953093 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1360/2715] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:50 2022] DELETE /identity/v3/users/e557fdf74add40dca1335f24242128dc => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:51.138153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd219a4b-4d5c-4543-8cd2-d2c6380485f0 service placement] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:51.149308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd219a4b-4d5c-4543-8cd2-d2c6380485f0 service placement] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:51.208880 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff27bc-f482-4f55-be86-c36392a5af49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:51.209742 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff27bc-f482-4f55-be86-c36392a5af49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:51.210159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff27bc-f482-4f55-be86-c36392a5af49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:51.210556 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff27bc-f482-4f55-be86-c36392a5af49 admin admin] PATH_INFO: `/v3/projects/d82dce8fbe2846debf5ee4efc1e74bdb` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:51.212410 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff27bc-f482-4f55-be86-c36392a5af49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d82dce8fbe2846debf5ee4efc1e74bdb)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:51.215443 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff27bc-f482-4f55-be86-c36392a5af49 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:51.273082 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-20ff27bc-f482-4f55-be86-c36392a5af49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd82dce8fbe2846debf5ee4efc1e74bdb', 'request_id': 'req-20ff27bc-f482-4f55-be86-c36392a5af49'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:51.278759 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-20ff27bc-f482-4f55-be86-c36392a5af49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd82dce8fbe2846debf5ee4efc1e74bdb', 'request_id': 'req-20ff27bc-f482-4f55-be86-c36392a5af49'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:51.285332 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-20ff27bc-f482-4f55-be86-c36392a5af49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd82dce8fbe2846debf5ee4efc1e74bdb', 'request_id': 'req-20ff27bc-f482-4f55-be86-c36392a5af49'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:51.300091 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-20ff27bc-f482-4f55-be86-c36392a5af49 admin admin] The token cache is being invalidate because project d82dce8fbe2846debf5ee4efc1e74bdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:51.300374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-20ff27bc-f482-4f55-be86-c36392a5af49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:51.301027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-20ff27bc-f482-4f55-be86-c36392a5af49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:51.302579 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1356/2716] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:51 2022] DELETE /identity/v3/projects/d82dce8fbe2846debf5ee4efc1e74bdb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:51.306743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd820be-f601-4541-aa8c-4e4761c29851 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:51.308250 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ebd820be-f601-4541-aa8c-4e4761c29851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:51.321867 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd820be-f601-4541-aa8c-4e4761c29851 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:51.380405 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e482ee7-3460-4360-97b4-28aff852b0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:51.381036 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e482ee7-3460-4360-97b4-28aff852b0e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:51.381308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e482ee7-3460-4360-97b4-28aff852b0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:51.381552 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e482ee7-3460-4360-97b4-28aff852b0e7 admin admin] PATH_INFO: `/v3/users/572b77ffe3114453a45d69e3a6a915c0` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:51.382709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e482ee7-3460-4360-97b4-28aff852b0e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=572b77ffe3114453a45d69e3a6a915c0)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:51.384695 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e482ee7-3460-4360-97b4-28aff852b0e7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:51.408137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:51.439214 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0e482ee7-3460-4360-97b4-28aff852b0e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '572b77ffe3114453a45d69e3a6a915c0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:51.448270 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0e482ee7-3460-4360-97b4-28aff852b0e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '572b77ffe3114453a45d69e3a6a915c0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:51.457789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0e482ee7-3460-4360-97b4-28aff852b0e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '572b77ffe3114453a45d69e3a6a915c0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:51.458394 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0e482ee7-3460-4360-97b4-28aff852b0e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '572b77ffe3114453a45d69e3a6a915c0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:51.472439 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0e482ee7-3460-4360-97b4-28aff852b0e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '572b77ffe3114453a45d69e3a6a915c0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:51.479247 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1361/2717] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:51 2022] DELETE /identity/v3/users/572b77ffe3114453a45d69e3a6a915c0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:51.672566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff27bc-f482-4f55-be86-c36392a5af49 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:51.674036 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-20ff27bc-f482-4f55-be86-c36392a5af49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:51.688993 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff27bc-f482-4f55-be86-c36392a5af49 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:51.707358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:51.749699 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0fae5-b70e-497e-8382-76f5428543c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:51.750338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0fae5-b70e-497e-8382-76f5428543c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:51.750615 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0fae5-b70e-497e-8382-76f5428543c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:51.750883 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0fae5-b70e-497e-8382-76f5428543c6 admin admin] PATH_INFO: `/v3/projects/233672949cdd4636b2f95a6889c66058` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:51.752059 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca0fae5-b70e-497e-8382-76f5428543c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=233672949cdd4636b2f95a6889c66058)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:51.753978 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca0fae5-b70e-497e-8382-76f5428543c6 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:51.805515 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eca0fae5-b70e-497e-8382-76f5428543c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '233672949cdd4636b2f95a6889c66058', 'request_id': 'req-eca0fae5-b70e-497e-8382-76f5428543c6'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:51.810970 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eca0fae5-b70e-497e-8382-76f5428543c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '233672949cdd4636b2f95a6889c66058', 'request_id': 'req-eca0fae5-b70e-497e-8382-76f5428543c6'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:51.817239 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eca0fae5-b70e-497e-8382-76f5428543c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '233672949cdd4636b2f95a6889c66058', 'request_id': 'req-eca0fae5-b70e-497e-8382-76f5428543c6'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:51.831902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eca0fae5-b70e-497e-8382-76f5428543c6 admin admin] The token cache is being invalidate because project 233672949cdd4636b2f95a6889c66058 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:51.832207 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eca0fae5-b70e-497e-8382-76f5428543c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:51.833007 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-eca0fae5-b70e-497e-8382-76f5428543c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:51.834546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1357/2718] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:51 2022] DELETE /identity/v3/projects/233672949cdd4636b2f95a6889c66058 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:51.839588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247aab5-2e2f-46a1-8bb8-4816c4d5ba0c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:51.840113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247aab5-2e2f-46a1-8bb8-4816c4d5ba0c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:51.840230 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247aab5-2e2f-46a1-8bb8-4816c4d5ba0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:51.861136 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-6247aab5-2e2f-46a1-8bb8-4816c4d5ba0c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:51.862378 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6247aab5-2e2f-46a1-8bb8-4816c4d5ba0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:51.927405 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1362/2719] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:51.935601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74da9a6-3924-4602-af85-bcce89fc85b5 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:51.935878 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74da9a6-3924-4602-af85-bcce89fc85b5 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:51.936182 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74da9a6-3924-4602-af85-bcce89fc85b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:51.957581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-f74da9a6-3924-4602-af85-bcce89fc85b5 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:51.958769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f74da9a6-3924-4602-af85-bcce89fc85b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:51.968199 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1358/2720] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:51.972938 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247aab5-2e2f-46a1-8bb8-4816c4d5ba0c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:51.983139 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247aab5-2e2f-46a1-8bb8-4816c4d5ba0c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:51.988643 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f744a72-f86e-4e80-8472-c374a16462db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:51.989242 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f744a72-f86e-4e80-8472-c374a16462db admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:51.989518 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f744a72-f86e-4e80-8472-c374a16462db admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:51.989788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f744a72-f86e-4e80-8472-c374a16462db admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:51.990436 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f744a72-f86e-4e80-8472-c374a16462db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:51.990737 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f744a72-f86e-4e80-8472-c374a16462db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:51.992412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f744a72-f86e-4e80-8472-c374a16462db admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:52.004286 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1363/2721] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:52.008926 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74da9a6-3924-4602-af85-bcce89fc85b5 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:52.018875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74da9a6-3924-4602-af85-bcce89fc85b5 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:52.024857 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fb24c-5ca2-464e-b8a5-63ea3766fa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:52.025469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fb24c-5ca2-464e-b8a5-63ea3766fa21 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:52.025745 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fb24c-5ca2-464e-b8a5-63ea3766fa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:52.026013 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fb24c-5ca2-464e-b8a5-63ea3766fa21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:52.026652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5fb24c-5ca2-464e-b8a5-63ea3766fa21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:52.028601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5fb24c-5ca2-464e-b8a5-63ea3766fa21 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:52.061288 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1359/2722] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:52 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:52.065957 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f744a72-f86e-4e80-8472-c374a16462db admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:52.076751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f744a72-f86e-4e80-8472-c374a16462db admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:52.134820 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c3541-0976-4a7a-aacc-a5d35dd0cdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:52.135431 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c3541-0976-4a7a-aacc-a5d35dd0cdc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:52.135705 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c3541-0976-4a7a-aacc-a5d35dd0cdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:52.135972 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c3541-0976-4a7a-aacc-a5d35dd0cdc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:52.136636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c3541-0976-4a7a-aacc-a5d35dd0cdc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:52.138543 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c3541-0976-4a7a-aacc-a5d35dd0cdc0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:52.161939 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1364/2723] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:52 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:52.166140 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fb24c-5ca2-464e-b8a5-63ea3766fa21 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:52.177113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fb24c-5ca2-464e-b8a5-63ea3766fa21 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:52.182646 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01987c34-fe98-414d-ba19-12e0ee0d3ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:52.183247 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01987c34-fe98-414d-ba19-12e0ee0d3ece admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:52.183528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01987c34-fe98-414d-ba19-12e0ee0d3ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:52.183784 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01987c34-fe98-414d-ba19-12e0ee0d3ece admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:52.184478 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01987c34-fe98-414d-ba19-12e0ee0d3ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:52.186131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01987c34-fe98-414d-ba19-12e0ee0d3ece admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:52.196485 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1360/2724] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:52.200886 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c3541-0976-4a7a-aacc-a5d35dd0cdc0 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:52.211898 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c3541-0976-4a7a-aacc-a5d35dd0cdc0 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:52.217297 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c59ed8-1b03-499d-b812-f49f6cdad75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:52.217918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c59ed8-1b03-499d-b812-f49f6cdad75c admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:52.218171 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c59ed8-1b03-499d-b812-f49f6cdad75c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:52.218452 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c59ed8-1b03-499d-b812-f49f6cdad75c admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:52.219103 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c59ed8-1b03-499d-b812-f49f6cdad75c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:52.220756 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c59ed8-1b03-499d-b812-f49f6cdad75c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:52.232458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1365/2725] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:52.236864 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01987c34-fe98-414d-ba19-12e0ee0d3ece admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:52.247223 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01987c34-fe98-414d-ba19-12e0ee0d3ece admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:52.252075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bca89a-f8ee-4861-9b30-faf0d18cb74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:52.252706 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bca89a-f8ee-4861-9b30-faf0d18cb74a admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:52.253099 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bca89a-f8ee-4861-9b30-faf0d18cb74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:52.253375 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bca89a-f8ee-4861-9b30-faf0d18cb74a admin admin] PATH_INFO: `/v3/projects/5af4866084dc4236817020fd3424a3d3/users/876ada9aa3e84cb0865f55ff1c5f4723/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:52.399277 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bca89a-f8ee-4861-9b30-faf0d18cb74a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5af4866084dc4236817020fd3424a3d3, user_id=876ada9aa3e84cb0865f55ff1c5f4723, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:52.402449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bca89a-f8ee-4861-9b30-faf0d18cb74a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:52.418049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1361/2726] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:52 2022] PUT /identity/v3/projects/5af4866084dc4236817020fd3424a3d3/users/876ada9aa3e84cb0865f55ff1c5f4723/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:52.429540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49a51c-bc7f-4a37-9245-9c527f318bfd None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:52.429821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49a51c-bc7f-4a37-9245-9c527f318bfd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:52.430086 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49a51c-bc7f-4a37-9245-9c527f318bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:52.438547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:52.461083 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0e49a51c-bc7f-4a37-9245-9c527f318bfd None None] MFA Rules not processed for user `876ada9aa3e84cb0865f55ff1c5f4723`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:52.462387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0e49a51c-bc7f-4a37-9245-9c527f318bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:52.535056 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1366/2727] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:52.539974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bca89a-f8ee-4861-9b30-faf0d18cb74a admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:52.590685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:52.608596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bca89a-f8ee-4861-9b30-faf0d18cb74a admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:52.613756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e99156-d2dd-4876-a083-bcf49a6084bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:52.614358 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e99156-d2dd-4876-a083-bcf49a6084bf admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:52.614630 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e99156-d2dd-4876-a083-bcf49a6084bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:52.614896 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e99156-d2dd-4876-a083-bcf49a6084bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:52.615520 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e99156-d2dd-4876-a083-bcf49a6084bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:52.617426 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e99156-d2dd-4876-a083-bcf49a6084bf admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:52.645022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1362/2728] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:52 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:56:52.649721 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49a51c-bc7f-4a37-9245-9c527f318bfd None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:52.661905 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49a51c-bc7f-4a37-9245-9c527f318bfd None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:52.725157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f8112-ef68-451c-85c5-15461e08737b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:52.725657 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f8112-ef68-451c-85c5-15461e08737b admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:52.725931 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f8112-ef68-451c-85c5-15461e08737b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:52.726198 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f8112-ef68-451c-85c5-15461e08737b admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:52.726890 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f8112-ef68-451c-85c5-15461e08737b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:52.728817 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f8112-ef68-451c-85c5-15461e08737b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:52.756706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1367/2729] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:52 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:52.761036 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e99156-d2dd-4876-a083-bcf49a6084bf admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:52.772170 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e99156-d2dd-4876-a083-bcf49a6084bf admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:52.777691 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390165d6-92cb-4b15-9ed4-f03fba1b2df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:52.778275 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390165d6-92cb-4b15-9ed4-f03fba1b2df7 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:52.778547 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390165d6-92cb-4b15-9ed4-f03fba1b2df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:52.778818 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390165d6-92cb-4b15-9ed4-f03fba1b2df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:52.779448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390165d6-92cb-4b15-9ed4-f03fba1b2df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:52.781285 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390165d6-92cb-4b15-9ed4-f03fba1b2df7 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:52.793769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1363/2730] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:52.798404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f8112-ef68-451c-85c5-15461e08737b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:52.809654 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f8112-ef68-451c-85c5-15461e08737b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:52.814577 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875acb28-6534-4473-a106-a6383376ed87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:52.815246 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875acb28-6534-4473-a106-a6383376ed87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:52.815804 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875acb28-6534-4473-a106-a6383376ed87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:52.816113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875acb28-6534-4473-a106-a6383376ed87 admin admin] PATH_INFO: `/v3/projects/ef302be64f714d6d9028b5be21caf85f/users/34532617c80043e7b1caa48573008169/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:52.831372 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875acb28-6534-4473-a106-a6383376ed87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef302be64f714d6d9028b5be21caf85f, user_id=34532617c80043e7b1caa48573008169, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:52.834339 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875acb28-6534-4473-a106-a6383376ed87 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:52.851064 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1368/2731] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:52 2022] PUT /identity/v3/projects/ef302be64f714d6d9028b5be21caf85f/users/34532617c80043e7b1caa48573008169/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:52.855985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390165d6-92cb-4b15-9ed4-f03fba1b2df7 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:52.870697 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390165d6-92cb-4b15-9ed4-f03fba1b2df7 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:52.930305 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb11b17-4280-4241-bdef-e0fbc2ab5d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:52.930913 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb11b17-4280-4241-bdef-e0fbc2ab5d60 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:52.931184 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb11b17-4280-4241-bdef-e0fbc2ab5d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:52.931446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb11b17-4280-4241-bdef-e0fbc2ab5d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:52.932110 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb11b17-4280-4241-bdef-e0fbc2ab5d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:52.933692 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb11b17-4280-4241-bdef-e0fbc2ab5d60 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:52.943437 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1364/2732] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:52.947639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875acb28-6534-4473-a106-a6383376ed87 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:52.958149 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875acb28-6534-4473-a106-a6383376ed87 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:52.963035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5769ee-49ab-4be7-bd5d-1858b3318bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:52.963685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5769ee-49ab-4be7-bd5d-1858b3318bcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:52.963957 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5769ee-49ab-4be7-bd5d-1858b3318bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:52.964253 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5769ee-49ab-4be7-bd5d-1858b3318bcd admin admin] PATH_INFO: `/v3/projects/ef302be64f714d6d9028b5be21caf85f/users/34532617c80043e7b1caa48573008169/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:52.966132 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5769ee-49ab-4be7-bd5d-1858b3318bcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef302be64f714d6d9028b5be21caf85f, user_id=34532617c80043e7b1caa48573008169, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:52.968790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5769ee-49ab-4be7-bd5d-1858b3318bcd admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:52.979907 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1369/2733] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:52 2022] PUT /identity/v3/projects/ef302be64f714d6d9028b5be21caf85f/users/34532617c80043e7b1caa48573008169/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:53.050656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbbcab-5e09-49d4-b2e9-60bf6f38ab26 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:53.051061 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbbcab-5e09-49d4-b2e9-60bf6f38ab26 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:53.051448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbbcab-5e09-49d4-b2e9-60bf6f38ab26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:53.086490 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-48fbbcab-5e09-49d4-b2e9-60bf6f38ab26 None None] MFA Rules not processed for user `34532617c80043e7b1caa48573008169`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:53.087725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-48fbbcab-5e09-49d4-b2e9-60bf6f38ab26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:53.159563 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1365/2734] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:53.171119 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5769ee-49ab-4be7-bd5d-1858b3318bcd admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:53.173090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ea5769ee-49ab-4be7-bd5d-1858b3318bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:53.243312 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:53.250322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5769ee-49ab-4be7-bd5d-1858b3318bcd admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:53.254623 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a661a9-2da8-412a-b56c-c76c05ce8f5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:53.255238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a661a9-2da8-412a-b56c-c76c05ce8f5f service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:53.255511 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a661a9-2da8-412a-b56c-c76c05ce8f5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:53.255780 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a661a9-2da8-412a-b56c-c76c05ce8f5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:53.326622 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a661a9-2da8-412a-b56c-c76c05ce8f5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:53.328971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a661a9-2da8-412a-b56c-c76c05ce8f5f service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:53.339578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1370/2735] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:55.921340 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbbcab-5e09-49d4-b2e9-60bf6f38ab26 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:55.978041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:55.998519 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbbcab-5e09-49d4-b2e9-60bf6f38ab26 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:56.003436 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901da70-e69d-4fe6-8468-130447ba7a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:56.004107 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901da70-e69d-4fe6-8468-130447ba7a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:56.004367 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901da70-e69d-4fe6-8468-130447ba7a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:56.004633 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901da70-e69d-4fe6-8468-130447ba7a07 admin admin] PATH_INFO: `/v3/users/876ada9aa3e84cb0865f55ff1c5f4723` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:56.005869 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901da70-e69d-4fe6-8468-130447ba7a07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=876ada9aa3e84cb0865f55ff1c5f4723)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:56.007766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901da70-e69d-4fe6-8468-130447ba7a07 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:56.065932 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-1901da70-e69d-4fe6-8468-130447ba7a07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '876ada9aa3e84cb0865f55ff1c5f4723'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:56.066765 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-1901da70-e69d-4fe6-8468-130447ba7a07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '876ada9aa3e84cb0865f55ff1c5f4723'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:56.067983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-1901da70-e69d-4fe6-8468-130447ba7a07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '876ada9aa3e84cb0865f55ff1c5f4723'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:56.077512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-1901da70-e69d-4fe6-8468-130447ba7a07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '876ada9aa3e84cb0865f55ff1c5f4723'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:56.090366 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-1901da70-e69d-4fe6-8468-130447ba7a07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '876ada9aa3e84cb0865f55ff1c5f4723'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:56.104248 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1366/2736] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:55 2022] DELETE /identity/v3/users/876ada9aa3e84cb0865f55ff1c5f4723 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:56.109083 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64419a95-43f5-4657-9a99-a32227574035 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:56.109632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64419a95-43f5-4657-9a99-a32227574035 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:56.109632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64419a95-43f5-4657-9a99-a32227574035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:56.131723 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-64419a95-43f5-4657-9a99-a32227574035 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:56.132970 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-64419a95-43f5-4657-9a99-a32227574035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:56.198518 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1371/2737] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:56:56.210931 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901da70-e69d-4fe6-8468-130447ba7a07 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:56.213265 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-1901da70-e69d-4fe6-8468-130447ba7a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:56.285530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901da70-e69d-4fe6-8468-130447ba7a07 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:56.289738 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36ef36-e937-4bc2-8d1c-4de11580f63c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:56.290347 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36ef36-e937-4bc2-8d1c-4de11580f63c service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:56.290620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36ef36-e937-4bc2-8d1c-4de11580f63c service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:56.290884 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36ef36-e937-4bc2-8d1c-4de11580f63c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:56.302041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36ef36-e937-4bc2-8d1c-4de11580f63c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:56.303548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36ef36-e937-4bc2-8d1c-4de11580f63c service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:56.313684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1367/2738] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:56.637224 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64419a95-43f5-4657-9a99-a32227574035 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:56.639595 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-64419a95-43f5-4657-9a99-a32227574035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:56.652432 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64419a95-43f5-4657-9a99-a32227574035 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:56.657696 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261120e-67df-4e27-a36c-fd498b657894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:56.658453 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261120e-67df-4e27-a36c-fd498b657894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:56.658797 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261120e-67df-4e27-a36c-fd498b657894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:56.659071 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261120e-67df-4e27-a36c-fd498b657894 admin admin] PATH_INFO: `/v3/projects/5af4866084dc4236817020fd3424a3d3` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:56.660454 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261120e-67df-4e27-a36c-fd498b657894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5af4866084dc4236817020fd3424a3d3)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:56.662512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261120e-67df-4e27-a36c-fd498b657894 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:56.713545 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d261120e-67df-4e27-a36c-fd498b657894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5af4866084dc4236817020fd3424a3d3', 'request_id': 'req-d261120e-67df-4e27-a36c-fd498b657894'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:56.722760 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d261120e-67df-4e27-a36c-fd498b657894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5af4866084dc4236817020fd3424a3d3', 'request_id': 'req-d261120e-67df-4e27-a36c-fd498b657894'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:56.729353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d261120e-67df-4e27-a36c-fd498b657894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5af4866084dc4236817020fd3424a3d3', 'request_id': 'req-d261120e-67df-4e27-a36c-fd498b657894'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:56.741658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d261120e-67df-4e27-a36c-fd498b657894 admin admin] The token cache is being invalidate because project 5af4866084dc4236817020fd3424a3d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:56.741942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d261120e-67df-4e27-a36c-fd498b657894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:56.742630 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d261120e-67df-4e27-a36c-fd498b657894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:56.744183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1372/2739] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:56 2022] DELETE /identity/v3/projects/5af4866084dc4236817020fd3424a3d3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:56.749028 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36ef36-e937-4bc2-8d1c-4de11580f63c service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:56.761435 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36ef36-e937-4bc2-8d1c-4de11580f63c service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:56.820965 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e42f8d-63bf-418c-aace-e087c2e17c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:56.821629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e42f8d-63bf-418c-aace-e087c2e17c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:56.822503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e42f8d-63bf-418c-aace-e087c2e17c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:56.822804 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e42f8d-63bf-418c-aace-e087c2e17c45 admin admin] PATH_INFO: `/v3/users/34532617c80043e7b1caa48573008169` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:56.823966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e42f8d-63bf-418c-aace-e087c2e17c45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34532617c80043e7b1caa48573008169)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:56.825890 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e42f8d-63bf-418c-aace-e087c2e17c45 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:56.830928 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:56.877569 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-42e42f8d-63bf-418c-aace-e087c2e17c45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34532617c80043e7b1caa48573008169'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:56.878235 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-42e42f8d-63bf-418c-aace-e087c2e17c45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34532617c80043e7b1caa48573008169'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:56.878842 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-42e42f8d-63bf-418c-aace-e087c2e17c45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34532617c80043e7b1caa48573008169'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:56.888665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-42e42f8d-63bf-418c-aace-e087c2e17c45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34532617c80043e7b1caa48573008169'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:56.904505 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-42e42f8d-63bf-418c-aace-e087c2e17c45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34532617c80043e7b1caa48573008169'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:56.920383 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1368/2740] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:56 2022] DELETE /identity/v3/users/34532617c80043e7b1caa48573008169 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:57.205883 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261120e-67df-4e27-a36c-fd498b657894 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:57.207343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d261120e-67df-4e27-a36c-fd498b657894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:57.221175 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261120e-67df-4e27-a36c-fd498b657894 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:57.242375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:57.285535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:57.285535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:57.285908 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:57.286080 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835 admin admin] PATH_INFO: `/v3/projects/ef302be64f714d6d9028b5be21caf85f` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:57.287322 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef302be64f714d6d9028b5be21caf85f)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:57.289269 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:57.346063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef302be64f714d6d9028b5be21caf85f', 'request_id': 'req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:57.355158 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef302be64f714d6d9028b5be21caf85f', 'request_id': 'req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:57.361204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef302be64f714d6d9028b5be21caf85f', 'request_id': 'req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:57.372603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835 admin admin] The token cache is being invalidate because project ef302be64f714d6d9028b5be21caf85f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:57.372912 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:57.373626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:57.375224 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1373/2741] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:57 2022] DELETE /identity/v3/projects/ef302be64f714d6d9028b5be21caf85f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:06.914537 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=24,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:07.369511 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:24.071407 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e42f8d-63bf-418c-aace-e087c2e17c45 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:24.072834 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-42e42f8d-63bf-418c-aace-e087c2e17c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:24.154602 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e42f8d-63bf-418c-aace-e087c2e17c45 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:24.220400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c51cac-6490-4406-ac20-c65a87805fdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:24.221013 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c51cac-6490-4406-ac20-c65a87805fdc service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:24.221287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c51cac-6490-4406-ac20-c65a87805fdc service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:24.221550 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c51cac-6490-4406-ac20-c65a87805fdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:24.223552 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c3c51cac-6490-4406-ac20-c65a87805fdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:24.307929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c51cac-6490-4406-ac20-c65a87805fdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:24.310347 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c51cac-6490-4406-ac20-c65a87805fdc service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:24.331299 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1369/2742] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:34.328315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:38.245160 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:38.247157 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:38.265335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042b29b-bb54-4ad5-8a60-51a9a7b6f835 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:38.273470 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87e370-3d15-4a6d-8304-a5a8e5b8b154 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:38.274422 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87e370-3d15-4a6d-8304-a5a8e5b8b154 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:38.274784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87e370-3d15-4a6d-8304-a5a8e5b8b154 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:38.275245 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87e370-3d15-4a6d-8304-a5a8e5b8b154 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:38.277358 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ad87e370-3d15-4a6d-8304-a5a8e5b8b154 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:38.291479 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87e370-3d15-4a6d-8304-a5a8e5b8b154 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:38.293179 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87e370-3d15-4a6d-8304-a5a8e5b8b154 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:38.306279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1374/2743] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:38.444784 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c51cac-6490-4406-ac20-c65a87805fdc service nova] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:38.446294 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c3c51cac-6490-4406-ac20-c65a87805fdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:38.520310 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c51cac-6490-4406-ac20-c65a87805fdc service nova] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:38.526088 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f24171-0f04-4341-b692-1b7b45b93bb0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cdd6b1ee9f4e97beb9bb037e446c8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:38.526703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f24171-0f04-4341-b692-1b7b45b93bb0 service placement] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:38.526972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f24171-0f04-4341-b692-1b7b45b93bb0 service placement] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:38.527216 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f24171-0f04-4341-b692-1b7b45b93bb0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:38.529866 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-27f24171-0f04-4341-b692-1b7b45b93bb0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:38.542551 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f24171-0f04-4341-b692-1b7b45b93bb0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:38.544495 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f24171-0f04-4341-b692-1b7b45b93bb0 service placement] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:38.565918 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1370/2744] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:44.377148 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87e370-3d15-4a6d-8304-a5a8e5b8b154 service nova] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:44.378751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ad87e370-3d15-4a6d-8304-a5a8e5b8b154 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:44.442245 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87e370-3d15-4a6d-8304-a5a8e5b8b154 service nova] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:44.447585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2610832-a9ff-4208-beb8-f9d78e7c8a4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:44.448245 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2610832-a9ff-4208-beb8-f9d78e7c8a4f service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:44.448524 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2610832-a9ff-4208-beb8-f9d78e7c8a4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:44.448916 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2610832-a9ff-4208-beb8-f9d78e7c8a4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:44.451369 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d2610832-a9ff-4208-beb8-f9d78e7c8a4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:44.524078 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2610832-a9ff-4208-beb8-f9d78e7c8a4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:44.525556 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2610832-a9ff-4208-beb8-f9d78e7c8a4f service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:44.535547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1375/2745] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:47.891197 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f24171-0f04-4341-b692-1b7b45b93bb0 service placement] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:47.892766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-27f24171-0f04-4341-b692-1b7b45b93bb0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:47.914672 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f24171-0f04-4341-b692-1b7b45b93bb0 service placement] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:47.919980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f7422-ba9c-4ea0-b778-dd9c5d13ac31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:47.920656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f7422-ba9c-4ea0-b778-dd9c5d13ac31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:47.920928 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f7422-ba9c-4ea0-b778-dd9c5d13ac31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:47.921197 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f7422-ba9c-4ea0-b778-dd9c5d13ac31 admin admin] PATH_INFO: `/v3/users/f72e1a846e864aa8b00edd8ae1a948ff` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:47.922369 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f7422-ba9c-4ea0-b778-dd9c5d13ac31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f72e1a846e864aa8b00edd8ae1a948ff)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:47.924666 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f7422-ba9c-4ea0-b778-dd9c5d13ac31 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:47.974139 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-757f7422-ba9c-4ea0-b778-dd9c5d13ac31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f72e1a846e864aa8b00edd8ae1a948ff'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:47.974790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-757f7422-ba9c-4ea0-b778-dd9c5d13ac31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f72e1a846e864aa8b00edd8ae1a948ff'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:47.975462 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-757f7422-ba9c-4ea0-b778-dd9c5d13ac31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f72e1a846e864aa8b00edd8ae1a948ff'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:47.984326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-757f7422-ba9c-4ea0-b778-dd9c5d13ac31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f72e1a846e864aa8b00edd8ae1a948ff'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:47.996424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-757f7422-ba9c-4ea0-b778-dd9c5d13ac31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f72e1a846e864aa8b00edd8ae1a948ff'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:48.008992 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1371/2746] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:47 2022] DELETE /identity/v3/users/f72e1a846e864aa8b00edd8ae1a948ff => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:48.203827 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2610832-a9ff-4208-beb8-f9d78e7c8a4f service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:48.215807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2610832-a9ff-4208-beb8-f9d78e7c8a4f service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:48.281167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c392e-6d4c-4612-9207-6b2a62b68393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:48.281808 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c392e-6d4c-4612-9207-6b2a62b68393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:48.282151 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c392e-6d4c-4612-9207-6b2a62b68393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:48.282340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c392e-6d4c-4612-9207-6b2a62b68393 admin admin] PATH_INFO: `/v3/projects/11b0254ba7d0421e9d9214a1c400466a` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:48.283517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c392e-6d4c-4612-9207-6b2a62b68393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11b0254ba7d0421e9d9214a1c400466a)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:48.285498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c392e-6d4c-4612-9207-6b2a62b68393 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:48.327861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:48.337070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ca3c392e-6d4c-4612-9207-6b2a62b68393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11b0254ba7d0421e9d9214a1c400466a', 'request_id': 'req-ca3c392e-6d4c-4612-9207-6b2a62b68393'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:48.346089 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ca3c392e-6d4c-4612-9207-6b2a62b68393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11b0254ba7d0421e9d9214a1c400466a', 'request_id': 'req-ca3c392e-6d4c-4612-9207-6b2a62b68393'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:48.352901 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ca3c392e-6d4c-4612-9207-6b2a62b68393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11b0254ba7d0421e9d9214a1c400466a', 'request_id': 'req-ca3c392e-6d4c-4612-9207-6b2a62b68393'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:48.364910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ca3c392e-6d4c-4612-9207-6b2a62b68393 admin admin] The token cache is being invalidate because project 11b0254ba7d0421e9d9214a1c400466a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:48.365192 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ca3c392e-6d4c-4612-9207-6b2a62b68393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:48.366376 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ca3c392e-6d4c-4612-9207-6b2a62b68393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:48.367815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1376/2747] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:48 2022] DELETE /identity/v3/projects/11b0254ba7d0421e9d9214a1c400466a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:48.373040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c18e8-6c2e-44f5-8412-01bd4818e979 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:48.373326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c18e8-6c2e-44f5-8412-01bd4818e979 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:48.373598 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c18e8-6c2e-44f5-8412-01bd4818e979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:48.394589 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-910c18e8-6c2e-44f5-8412-01bd4818e979 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:48.395981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-910c18e8-6c2e-44f5-8412-01bd4818e979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:48.473817 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1372/2748] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:57:48.483075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73538e-6509-4521-a3d8-a5306d9886b2 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:48.483352 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73538e-6509-4521-a3d8-a5306d9886b2 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:48.483633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73538e-6509-4521-a3d8-a5306d9886b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:48.508567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ca73538e-6509-4521-a3d8-a5306d9886b2 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:48.509819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ca73538e-6509-4521-a3d8-a5306d9886b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:48.518073 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1377/2749] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:48.523012 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c18e8-6c2e-44f5-8412-01bd4818e979 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:48.535547 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:48.544046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c18e8-6c2e-44f5-8412-01bd4818e979 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:48.549995 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f854ea0-284a-45f4-961d-88e32fd29071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:48.550609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f854ea0-284a-45f4-961d-88e32fd29071 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:48.550884 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f854ea0-284a-45f4-961d-88e32fd29071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:48.551266 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f854ea0-284a-45f4-961d-88e32fd29071 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:48.551955 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f854ea0-284a-45f4-961d-88e32fd29071 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:57:48.552296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f854ea0-284a-45f4-961d-88e32fd29071 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:48.554563 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f854ea0-284a-45f4-961d-88e32fd29071 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:48.566557 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1373/2750] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:57:48.572131 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73538e-6509-4521-a3d8-a5306d9886b2 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:48.585967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73538e-6509-4521-a3d8-a5306d9886b2 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:48.590982 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d47162-94dd-43eb-b983-1c91368aee93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:48.591607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d47162-94dd-43eb-b983-1c91368aee93 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:48.591883 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d47162-94dd-43eb-b983-1c91368aee93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:48.592260 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d47162-94dd-43eb-b983-1c91368aee93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:48.592898 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d47162-94dd-43eb-b983-1c91368aee93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:48.594835 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d47162-94dd-43eb-b983-1c91368aee93 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:48.626894 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1378/2751] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:48 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:48.631244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f854ea0-284a-45f4-961d-88e32fd29071 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:48.641509 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f854ea0-284a-45f4-961d-88e32fd29071 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:48.699530 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1378d-5ffe-4f2f-8611-1929006c3e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:48.700172 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1378d-5ffe-4f2f-8611-1929006c3e76 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:48.700447 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1378d-5ffe-4f2f-8611-1929006c3e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:48.700711 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1378d-5ffe-4f2f-8611-1929006c3e76 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:48.701364 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1378d-5ffe-4f2f-8611-1929006c3e76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:48.703252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1378d-5ffe-4f2f-8611-1929006c3e76 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:48.725164 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1374/2752] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:48 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:48.729326 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d47162-94dd-43eb-b983-1c91368aee93 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:48.739809 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d47162-94dd-43eb-b983-1c91368aee93 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:48.746184 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081092e5-8403-4bc7-91a2-d0165cd25bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:48.746833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081092e5-8403-4bc7-91a2-d0165cd25bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:48.747160 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081092e5-8403-4bc7-91a2-d0165cd25bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:48.747530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081092e5-8403-4bc7-91a2-d0165cd25bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:48.748297 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081092e5-8403-4bc7-91a2-d0165cd25bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:48.749947 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081092e5-8403-4bc7-91a2-d0165cd25bc8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:48.761837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1379/2753] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:48.767243 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1378d-5ffe-4f2f-8611-1929006c3e76 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:48.779212 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1378d-5ffe-4f2f-8611-1929006c3e76 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:48.785342 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db183b-3077-40b4-8660-b3536c9159cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:48.785964 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db183b-3077-40b4-8660-b3536c9159cf admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:48.786236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db183b-3077-40b4-8660-b3536c9159cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:48.786502 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db183b-3077-40b4-8660-b3536c9159cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:48.787190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db183b-3077-40b4-8660-b3536c9159cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:48.789083 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db183b-3077-40b4-8660-b3536c9159cf admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:48.801132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1375/2754] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:48.806042 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081092e5-8403-4bc7-91a2-d0165cd25bc8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:48.818640 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081092e5-8403-4bc7-91a2-d0165cd25bc8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:48.825040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9946b-346d-48a1-a18d-c0541ce519ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:48.825732 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9946b-346d-48a1-a18d-c0541ce519ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:48.825981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9946b-346d-48a1-a18d-c0541ce519ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:48.826243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9946b-346d-48a1-a18d-c0541ce519ce admin admin] PATH_INFO: `/v3/projects/5a97325bf9a04b62b63255c9fd55d0cd/users/4f996cb45063449fb68596c0c8a08434/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:48.843624 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9946b-346d-48a1-a18d-c0541ce519ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a97325bf9a04b62b63255c9fd55d0cd, user_id=4f996cb45063449fb68596c0c8a08434, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:48.846366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9946b-346d-48a1-a18d-c0541ce519ce admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:48.862116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1380/2755] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:48 2022] PUT /identity/v3/projects/5a97325bf9a04b62b63255c9fd55d0cd/users/4f996cb45063449fb68596c0c8a08434/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:48.878690 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c59433-2efa-4a69-8d6e-5ceb8b76ba5a None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:48.878920 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c59433-2efa-4a69-8d6e-5ceb8b76ba5a None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:48.879157 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c59433-2efa-4a69-8d6e-5ceb8b76ba5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:48.908545 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-86c59433-2efa-4a69-8d6e-5ceb8b76ba5a None None] MFA Rules not processed for user `4f996cb45063449fb68596c0c8a08434`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:48.909766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-86c59433-2efa-4a69-8d6e-5ceb8b76ba5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:48.979875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1376/2756] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:57:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:48.987377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9946b-346d-48a1-a18d-c0541ce519ce admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:49.060933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9946b-346d-48a1-a18d-c0541ce519ce admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:49.066498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177dc87-3746-4a27-82dc-c8490c60d1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:49.067085 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177dc87-3746-4a27-82dc-c8490c60d1e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:49.067359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177dc87-3746-4a27-82dc-c8490c60d1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:49.067627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177dc87-3746-4a27-82dc-c8490c60d1e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:49.068279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177dc87-3746-4a27-82dc-c8490c60d1e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:49.070239 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177dc87-3746-4a27-82dc-c8490c60d1e8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:49.100522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1381/2757] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:48 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:49.105159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c59433-2efa-4a69-8d6e-5ceb8b76ba5a None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:49.116368 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c59433-2efa-4a69-8d6e-5ceb8b76ba5a None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:49.169141 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:49.177543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ff658-1290-4483-b1d5-951d89477ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:49.178211 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ff658-1290-4483-b1d5-951d89477ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:49.178642 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ff658-1290-4483-b1d5-951d89477ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:49.178977 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ff658-1290-4483-b1d5-951d89477ca9 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:49.179792 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ff658-1290-4483-b1d5-951d89477ca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:49.181954 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ff658-1290-4483-b1d5-951d89477ca9 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:49.205428 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1377/2758] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:49 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:49.210952 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177dc87-3746-4a27-82dc-c8490c60d1e8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:49.221611 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177dc87-3746-4a27-82dc-c8490c60d1e8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:49.226244 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b2b20-2895-42c5-967e-ab9361fbbd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:49.226841 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b2b20-2895-42c5-967e-ab9361fbbd20 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:49.227113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b2b20-2895-42c5-967e-ab9361fbbd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:49.227377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b2b20-2895-42c5-967e-ab9361fbbd20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:49.228033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b2b20-2895-42c5-967e-ab9361fbbd20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:49.229666 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b2b20-2895-42c5-967e-ab9361fbbd20 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:49.244562 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1382/2759] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:49.250276 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ff658-1290-4483-b1d5-951d89477ca9 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:49.261582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ff658-1290-4483-b1d5-951d89477ca9 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:49.267121 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf3ecd-5d5b-4d1a-9d6d-b41124b37a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:49.267812 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf3ecd-5d5b-4d1a-9d6d-b41124b37a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:49.268111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf3ecd-5d5b-4d1a-9d6d-b41124b37a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:49.268385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf3ecd-5d5b-4d1a-9d6d-b41124b37a0c admin admin] PATH_INFO: `/v3/projects/5c27dad5f29042bfba4df7aae515c830/users/67ce7b82a22d4d93a31a13950dfcf7d4/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:49.284639 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf3ecd-5d5b-4d1a-9d6d-b41124b37a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c27dad5f29042bfba4df7aae515c830, user_id=67ce7b82a22d4d93a31a13950dfcf7d4, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:49.287287 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf3ecd-5d5b-4d1a-9d6d-b41124b37a0c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:49.301982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1378/2760] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:49 2022] PUT /identity/v3/projects/5c27dad5f29042bfba4df7aae515c830/users/67ce7b82a22d4d93a31a13950dfcf7d4/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:49.306520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b2b20-2895-42c5-967e-ab9361fbbd20 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:49.316701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b2b20-2895-42c5-967e-ab9361fbbd20 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:49.353505 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:49.381369 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e6fcd-ab04-4cc9-9f19-3ec0b237e612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:49.382523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e6fcd-ab04-4cc9-9f19-3ec0b237e612 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:49.382523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e6fcd-ab04-4cc9-9f19-3ec0b237e612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:49.382767 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e6fcd-ab04-4cc9-9f19-3ec0b237e612 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:49.383446 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e6fcd-ab04-4cc9-9f19-3ec0b237e612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:49.385244 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e6fcd-ab04-4cc9-9f19-3ec0b237e612 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:49.395444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1383/2761] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:49.399811 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf3ecd-5d5b-4d1a-9d6d-b41124b37a0c admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:49.410668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf3ecd-5d5b-4d1a-9d6d-b41124b37a0c admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:49.416757 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604e14e-d95f-4177-9011-f4d3a6d229f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:49.417403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604e14e-d95f-4177-9011-f4d3a6d229f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:49.417729 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604e14e-d95f-4177-9011-f4d3a6d229f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:49.417996 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604e14e-d95f-4177-9011-f4d3a6d229f2 admin admin] PATH_INFO: `/v3/projects/5c27dad5f29042bfba4df7aae515c830/users/67ce7b82a22d4d93a31a13950dfcf7d4/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:49.420200 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604e14e-d95f-4177-9011-f4d3a6d229f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c27dad5f29042bfba4df7aae515c830, user_id=67ce7b82a22d4d93a31a13950dfcf7d4, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:49.422782 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604e14e-d95f-4177-9011-f4d3a6d229f2 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:49.435134 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1379/2762] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:49 2022] PUT /identity/v3/projects/5c27dad5f29042bfba4df7aae515c830/users/67ce7b82a22d4d93a31a13950dfcf7d4/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:49.447547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60f8c3-d79b-4806-8e84-9b602313f2b5 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:49.447798 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60f8c3-d79b-4806-8e84-9b602313f2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:49.448128 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60f8c3-d79b-4806-8e84-9b602313f2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:49.478162 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-cc60f8c3-d79b-4806-8e84-9b602313f2b5 None None] MFA Rules not processed for user `67ce7b82a22d4d93a31a13950dfcf7d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:49.479376 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cc60f8c3-d79b-4806-8e84-9b602313f2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:49.558620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1384/2763] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:57:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:49.570868 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604e14e-d95f-4177-9011-f4d3a6d229f2 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:49.573081 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0604e14e-d95f-4177-9011-f4d3a6d229f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:49.640725 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604e14e-d95f-4177-9011-f4d3a6d229f2 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:49.645147 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b68eef-52c6-4888-8df3-cb73c6b7f802 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:49.645739 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b68eef-52c6-4888-8df3-cb73c6b7f802 service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:49.646006 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b68eef-52c6-4888-8df3-cb73c6b7f802 service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:49.646269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b68eef-52c6-4888-8df3-cb73c6b7f802 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:49.657104 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b68eef-52c6-4888-8df3-cb73c6b7f802 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:49.658540 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b68eef-52c6-4888-8df3-cb73c6b7f802 service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:49.668723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1380/2764] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:49.765966 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60f8c3-d79b-4806-8e84-9b602313f2b5 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:49.777317 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60f8c3-d79b-4806-8e84-9b602313f2b5 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:49.781346 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcc44c-a128-4bc0-b392-a20b2352f707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:49.781953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcc44c-a128-4bc0-b392-a20b2352f707 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:49.782229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcc44c-a128-4bc0-b392-a20b2352f707 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:49.782770 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcc44c-a128-4bc0-b392-a20b2352f707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:49.844675 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dcc44c-a128-4bc0-b392-a20b2352f707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:49.846124 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dcc44c-a128-4bc0-b392-a20b2352f707 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:49.855714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1385/2765] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:59.663863 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=44,keystone:INSERT=5 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:59.851137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=74 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:02.314656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b68eef-52c6-4888-8df3-cb73c6b7f802 service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:02.316258 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-50b68eef-52c6-4888-8df3-cb73c6b7f802 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:02.377314 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b68eef-52c6-4888-8df3-cb73c6b7f802 service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:02.381186 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d34266-77f6-44ef-85b7-25d32a3eedd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:02.383338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d34266-77f6-44ef-85b7-25d32a3eedd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:02.383681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d34266-77f6-44ef-85b7-25d32a3eedd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:02.384135 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d34266-77f6-44ef-85b7-25d32a3eedd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:02.386956 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-26d34266-77f6-44ef-85b7-25d32a3eedd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:02.482790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d34266-77f6-44ef-85b7-25d32a3eedd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:02.484255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d34266-77f6-44ef-85b7-25d32a3eedd9 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:02.504198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1381/2766] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:12.501054 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=61 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:18.102161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcc44c-a128-4bc0-b392-a20b2352f707 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:18.103792 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d3dcc44c-a128-4bc0-b392-a20b2352f707 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:18.117584 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcc44c-a128-4bc0-b392-a20b2352f707 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:18.121424 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd4899-4947-4561-a647-2ca848e86b62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:18.122029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd4899-4947-4561-a647-2ca848e86b62 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:18.122299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd4899-4947-4561-a647-2ca848e86b62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:18.122563 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd4899-4947-4561-a647-2ca848e86b62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:18.124289 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f7fd4899-4947-4561-a647-2ca848e86b62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:18.190470 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd4899-4947-4561-a647-2ca848e86b62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:18.191970 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd4899-4947-4561-a647-2ca848e86b62 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:18.202138 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1386/2767] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:21.218385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d34266-77f6-44ef-85b7-25d32a3eedd9 service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:21.219876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-26d34266-77f6-44ef-85b7-25d32a3eedd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:21.307121 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d34266-77f6-44ef-85b7-25d32a3eedd9 service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:21.312627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc515be6-c6e0-40cf-a57c-8be64d66d1c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cdd6b1ee9f4e97beb9bb037e446c8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:21.313402 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc515be6-c6e0-40cf-a57c-8be64d66d1c4 service placement] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:21.313678 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc515be6-c6e0-40cf-a57c-8be64d66d1c4 service placement] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:21.313941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc515be6-c6e0-40cf-a57c-8be64d66d1c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:21.315694 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-dc515be6-c6e0-40cf-a57c-8be64d66d1c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:21.329258 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc515be6-c6e0-40cf-a57c-8be64d66d1c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:21.330881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc515be6-c6e0-40cf-a57c-8be64d66d1c4 service placement] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:21.343492 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1382/2768] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:58:24.590513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd4899-4947-4561-a647-2ca848e86b62 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:24.592223 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f7fd4899-4947-4561-a647-2ca848e86b62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:24.611488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd4899-4947-4561-a647-2ca848e86b62 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:24.630112 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12001f8a-f6bf-4e6a-920c-3055563c9bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:24.631258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12001f8a-f6bf-4e6a-920c-3055563c9bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:24.631533 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12001f8a-f6bf-4e6a-920c-3055563c9bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:24.632788 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12001f8a-f6bf-4e6a-920c-3055563c9bd8 admin admin] PATH_INFO: `/v3/users/06ab23cfa18e47c9871c6f451b871ef2` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:24.634351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12001f8a-f6bf-4e6a-920c-3055563c9bd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06ab23cfa18e47c9871c6f451b871ef2)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:24.636429 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12001f8a-f6bf-4e6a-920c-3055563c9bd8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:24.699016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-12001f8a-f6bf-4e6a-920c-3055563c9bd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06ab23cfa18e47c9871c6f451b871ef2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.707735 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-12001f8a-f6bf-4e6a-920c-3055563c9bd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06ab23cfa18e47c9871c6f451b871ef2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.718223 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-12001f8a-f6bf-4e6a-920c-3055563c9bd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06ab23cfa18e47c9871c6f451b871ef2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.718959 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-12001f8a-f6bf-4e6a-920c-3055563c9bd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06ab23cfa18e47c9871c6f451b871ef2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.736523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-12001f8a-f6bf-4e6a-920c-3055563c9bd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06ab23cfa18e47c9871c6f451b871ef2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.742709 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1387/2769] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:24 2022] DELETE /identity/v3/users/06ab23cfa18e47c9871c6f451b871ef2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:24.925117 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc515be6-c6e0-40cf-a57c-8be64d66d1c4 service placement] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:24.939824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc515be6-c6e0-40cf-a57c-8be64d66d1c4 service placement] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:25.003459 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734b9cd-9bfb-402c-95b6-3dc1231f6627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:25.004264 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734b9cd-9bfb-402c-95b6-3dc1231f6627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.004380 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734b9cd-9bfb-402c-95b6-3dc1231f6627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.004685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734b9cd-9bfb-402c-95b6-3dc1231f6627 admin admin] PATH_INFO: `/v3/projects/86111e1d9f5b485abf57231b21b3a5e9` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.005773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734b9cd-9bfb-402c-95b6-3dc1231f6627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86111e1d9f5b485abf57231b21b3a5e9)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:25.007656 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734b9cd-9bfb-402c-95b6-3dc1231f6627 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:25.061632 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4734b9cd-9bfb-402c-95b6-3dc1231f6627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86111e1d9f5b485abf57231b21b3a5e9', 'request_id': 'req-4734b9cd-9bfb-402c-95b6-3dc1231f6627'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.067936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4734b9cd-9bfb-402c-95b6-3dc1231f6627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86111e1d9f5b485abf57231b21b3a5e9', 'request_id': 'req-4734b9cd-9bfb-402c-95b6-3dc1231f6627'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.075648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4734b9cd-9bfb-402c-95b6-3dc1231f6627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86111e1d9f5b485abf57231b21b3a5e9', 'request_id': 'req-4734b9cd-9bfb-402c-95b6-3dc1231f6627'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.091053 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4734b9cd-9bfb-402c-95b6-3dc1231f6627 admin admin] The token cache is being invalidate because project 86111e1d9f5b485abf57231b21b3a5e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:25.091348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4734b9cd-9bfb-402c-95b6-3dc1231f6627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.091975 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4734b9cd-9bfb-402c-95b6-3dc1231f6627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.093618 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1383/2770] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:24 2022] DELETE /identity/v3/projects/86111e1d9f5b485abf57231b21b3a5e9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:25.097766 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12001f8a-f6bf-4e6a-920c-3055563c9bd8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:25.099198 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-12001f8a-f6bf-4e6a-920c-3055563c9bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:25.114182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12001f8a-f6bf-4e6a-920c-3055563c9bd8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:25.175164 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf5ff1-0102-4d6e-8cb0-fe002fe2ae79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:25.175814 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf5ff1-0102-4d6e-8cb0-fe002fe2ae79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.176120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf5ff1-0102-4d6e-8cb0-fe002fe2ae79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.181404 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf5ff1-0102-4d6e-8cb0-fe002fe2ae79 admin admin] PATH_INFO: `/v3/users/b0353dcc5d924fc2bb594d2d941ee40b` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.192836 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:25.198918 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf5ff1-0102-4d6e-8cb0-fe002fe2ae79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0353dcc5d924fc2bb594d2d941ee40b)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:25.201483 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf5ff1-0102-4d6e-8cb0-fe002fe2ae79 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:25.258888 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c6bf5ff1-0102-4d6e-8cb0-fe002fe2ae79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0353dcc5d924fc2bb594d2d941ee40b'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.266566 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c6bf5ff1-0102-4d6e-8cb0-fe002fe2ae79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0353dcc5d924fc2bb594d2d941ee40b'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.275359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c6bf5ff1-0102-4d6e-8cb0-fe002fe2ae79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0353dcc5d924fc2bb594d2d941ee40b'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.276498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c6bf5ff1-0102-4d6e-8cb0-fe002fe2ae79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0353dcc5d924fc2bb594d2d941ee40b'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.290018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c6bf5ff1-0102-4d6e-8cb0-fe002fe2ae79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0353dcc5d924fc2bb594d2d941ee40b'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.296025 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1388/2771] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:25 2022] DELETE /identity/v3/users/b0353dcc5d924fc2bb594d2d941ee40b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:25.486514 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734b9cd-9bfb-402c-95b6-3dc1231f6627 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:25.487989 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4734b9cd-9bfb-402c-95b6-3dc1231f6627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:25.503676 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734b9cd-9bfb-402c-95b6-3dc1231f6627 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:25.523084 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:25.567528 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4652cf-177b-438c-bdcb-68bad5436fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:25.568183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4652cf-177b-438c-bdcb-68bad5436fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.568461 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4652cf-177b-438c-bdcb-68bad5436fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.568732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4652cf-177b-438c-bdcb-68bad5436fb5 admin admin] PATH_INFO: `/v3/projects/e5f81b194ec04f058d62723478ba875a` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.580365 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4652cf-177b-438c-bdcb-68bad5436fb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5f81b194ec04f058d62723478ba875a)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:25.582445 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4652cf-177b-438c-bdcb-68bad5436fb5 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:25.636703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cf4652cf-177b-438c-bdcb-68bad5436fb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5f81b194ec04f058d62723478ba875a', 'request_id': 'req-cf4652cf-177b-438c-bdcb-68bad5436fb5'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.642195 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cf4652cf-177b-438c-bdcb-68bad5436fb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5f81b194ec04f058d62723478ba875a', 'request_id': 'req-cf4652cf-177b-438c-bdcb-68bad5436fb5'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.649225 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cf4652cf-177b-438c-bdcb-68bad5436fb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5f81b194ec04f058d62723478ba875a', 'request_id': 'req-cf4652cf-177b-438c-bdcb-68bad5436fb5'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.663731 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cf4652cf-177b-438c-bdcb-68bad5436fb5 admin admin] The token cache is being invalidate because project e5f81b194ec04f058d62723478ba875a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:25.664020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cf4652cf-177b-438c-bdcb-68bad5436fb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.664717 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-cf4652cf-177b-438c-bdcb-68bad5436fb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.666213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1384/2772] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:25 2022] DELETE /identity/v3/projects/e5f81b194ec04f058d62723478ba875a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:35.222424 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf5ff1-0102-4d6e-8cb0-fe002fe2ae79 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:35.223943 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c6bf5ff1-0102-4d6e-8cb0-fe002fe2ae79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:35.237255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf5ff1-0102-4d6e-8cb0-fe002fe2ae79 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:35.299274 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b191f8-f24c-4287-8914-75de556e87ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:35.299925 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b191f8-f24c-4287-8914-75de556e87ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:35.300237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b191f8-f24c-4287-8914-75de556e87ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:35.300506 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b191f8-f24c-4287-8914-75de556e87ba admin admin] PATH_INFO: `/v3/users/4f996cb45063449fb68596c0c8a08434` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:35.301706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b191f8-f24c-4287-8914-75de556e87ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f996cb45063449fb68596c0c8a08434)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:35.303675 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b191f8-f24c-4287-8914-75de556e87ba admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:35.356958 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-59b191f8-f24c-4287-8914-75de556e87ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f996cb45063449fb68596c0c8a08434'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:35.365040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-59b191f8-f24c-4287-8914-75de556e87ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f996cb45063449fb68596c0c8a08434'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:35.373674 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-59b191f8-f24c-4287-8914-75de556e87ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f996cb45063449fb68596c0c8a08434'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:35.374364 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-59b191f8-f24c-4287-8914-75de556e87ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f996cb45063449fb68596c0c8a08434'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:35.386234 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:35.388965 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-59b191f8-f24c-4287-8914-75de556e87ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f996cb45063449fb68596c0c8a08434'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:35.397368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1389/2773] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:35 2022] DELETE /identity/v3/users/4f996cb45063449fb68596c0c8a08434 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:35.402921 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d5a61-3b81-4c44-adcf-15618b3211c0 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:35.403439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d5a61-3b81-4c44-adcf-15618b3211c0 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:35.403439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d5a61-3b81-4c44-adcf-15618b3211c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:35.424595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-2d1d5a61-3b81-4c44-adcf-15618b3211c0 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:35.425826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2d1d5a61-3b81-4c44-adcf-15618b3211c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:35.497600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1385/2774] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:35.507700 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b191f8-f24c-4287-8914-75de556e87ba admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:35.509216 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-59b191f8-f24c-4287-8914-75de556e87ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:35.575626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b191f8-f24c-4287-8914-75de556e87ba admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:35.579616 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e822b5-5d6c-4e4c-b155-f3329cf97581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:35.580265 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e822b5-5d6c-4e4c-b155-f3329cf97581 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:35.580537 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e822b5-5d6c-4e4c-b155-f3329cf97581 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:35.580799 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e822b5-5d6c-4e4c-b155-f3329cf97581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:35.590429 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e822b5-5d6c-4e4c-b155-f3329cf97581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:35.591790 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e822b5-5d6c-4e4c-b155-f3329cf97581 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:35.601827 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1390/2775] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:35.892256 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d5a61-3b81-4c44-adcf-15618b3211c0 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:35.893941 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2d1d5a61-3b81-4c44-adcf-15618b3211c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:35.910004 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d5a61-3b81-4c44-adcf-15618b3211c0 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:35.916384 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda331ff-64a1-4ac3-9590-2ba582b0342d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:35.917004 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda331ff-64a1-4ac3-9590-2ba582b0342d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:35.917320 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda331ff-64a1-4ac3-9590-2ba582b0342d admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:35.917552 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda331ff-64a1-4ac3-9590-2ba582b0342d admin admin] PATH_INFO: `/v3/projects/5a97325bf9a04b62b63255c9fd55d0cd` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:35.918845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda331ff-64a1-4ac3-9590-2ba582b0342d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a97325bf9a04b62b63255c9fd55d0cd)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:35.920820 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda331ff-64a1-4ac3-9590-2ba582b0342d admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:35.966244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:35.977797 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-fda331ff-64a1-4ac3-9590-2ba582b0342d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a97325bf9a04b62b63255c9fd55d0cd', 'request_id': 'req-fda331ff-64a1-4ac3-9590-2ba582b0342d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:35.983983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-fda331ff-64a1-4ac3-9590-2ba582b0342d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a97325bf9a04b62b63255c9fd55d0cd', 'request_id': 'req-fda331ff-64a1-4ac3-9590-2ba582b0342d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:35.990438 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-fda331ff-64a1-4ac3-9590-2ba582b0342d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a97325bf9a04b62b63255c9fd55d0cd', 'request_id': 'req-fda331ff-64a1-4ac3-9590-2ba582b0342d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.006131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-fda331ff-64a1-4ac3-9590-2ba582b0342d admin admin] The token cache is being invalidate because project 5a97325bf9a04b62b63255c9fd55d0cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:36.006469 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-fda331ff-64a1-4ac3-9590-2ba582b0342d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.007218 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-fda331ff-64a1-4ac3-9590-2ba582b0342d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.008793 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1386/2776] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:35 2022] DELETE /identity/v3/projects/5a97325bf9a04b62b63255c9fd55d0cd => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:36.012981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e822b5-5d6c-4e4c-b155-f3329cf97581 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:36.025138 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e822b5-5d6c-4e4c-b155-f3329cf97581 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:36.083744 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac4300-ccfc-4112-8d7f-7f5811547e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:36.084394 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac4300-ccfc-4112-8d7f-7f5811547e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:36.084671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac4300-ccfc-4112-8d7f-7f5811547e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:36.084937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac4300-ccfc-4112-8d7f-7f5811547e7c admin admin] PATH_INFO: `/v3/users/67ce7b82a22d4d93a31a13950dfcf7d4` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:36.086058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac4300-ccfc-4112-8d7f-7f5811547e7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=67ce7b82a22d4d93a31a13950dfcf7d4)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:36.088023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac4300-ccfc-4112-8d7f-7f5811547e7c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:36.136459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f5ac4300-ccfc-4112-8d7f-7f5811547e7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67ce7b82a22d4d93a31a13950dfcf7d4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.147684 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f5ac4300-ccfc-4112-8d7f-7f5811547e7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67ce7b82a22d4d93a31a13950dfcf7d4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.158102 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f5ac4300-ccfc-4112-8d7f-7f5811547e7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67ce7b82a22d4d93a31a13950dfcf7d4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.158767 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f5ac4300-ccfc-4112-8d7f-7f5811547e7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67ce7b82a22d4d93a31a13950dfcf7d4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.170554 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f5ac4300-ccfc-4112-8d7f-7f5811547e7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67ce7b82a22d4d93a31a13950dfcf7d4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.177104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1391/2777] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:36 2022] DELETE /identity/v3/users/67ce7b82a22d4d93a31a13950dfcf7d4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:36.467461 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda331ff-64a1-4ac3-9590-2ba582b0342d admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:36.469285 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-fda331ff-64a1-4ac3-9590-2ba582b0342d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:36.485951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda331ff-64a1-4ac3-9590-2ba582b0342d admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:36.546538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02158152-9638-4f71-9b1b-9f45118782eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:36.547361 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02158152-9638-4f71-9b1b-9f45118782eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:36.547975 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02158152-9638-4f71-9b1b-9f45118782eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:36.548271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02158152-9638-4f71-9b1b-9f45118782eb admin admin] PATH_INFO: `/v3/projects/5c27dad5f29042bfba4df7aae515c830` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:36.549406 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02158152-9638-4f71-9b1b-9f45118782eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c27dad5f29042bfba4df7aae515c830)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:36.551283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02158152-9638-4f71-9b1b-9f45118782eb admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:36.608924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-02158152-9638-4f71-9b1b-9f45118782eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c27dad5f29042bfba4df7aae515c830', 'request_id': 'req-02158152-9638-4f71-9b1b-9f45118782eb'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.615603 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-02158152-9638-4f71-9b1b-9f45118782eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c27dad5f29042bfba4df7aae515c830', 'request_id': 'req-02158152-9638-4f71-9b1b-9f45118782eb'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.622386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-02158152-9638-4f71-9b1b-9f45118782eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c27dad5f29042bfba4df7aae515c830', 'request_id': 'req-02158152-9638-4f71-9b1b-9f45118782eb'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.637655 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-02158152-9638-4f71-9b1b-9f45118782eb admin admin] The token cache is being invalidate because project 5c27dad5f29042bfba4df7aae515c830 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:36.637946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-02158152-9638-4f71-9b1b-9f45118782eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.638692 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-02158152-9638-4f71-9b1b-9f45118782eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.640419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1387/2778] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:36 2022] DELETE /identity/v3/projects/5c27dad5f29042bfba4df7aae515c830 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:36.645485 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaacb64-042b-4d5b-b619-9909b188feaa None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:36.646007 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaacb64-042b-4d5b-b619-9909b188feaa None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:36.646125 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaacb64-042b-4d5b-b619-9909b188feaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:36.666202 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0eaacb64-042b-4d5b-b619-9909b188feaa None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:36.667378 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0eaacb64-042b-4d5b-b619-9909b188feaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:36.678832 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:36.731438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1392/2779] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:36.740764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb8b99-6cc9-460a-ab0f-f0eb03a3519c None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:36.741125 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb8b99-6cc9-460a-ab0f-f0eb03a3519c None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:36.741507 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb8b99-6cc9-460a-ab0f-f0eb03a3519c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:36.763346 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-80cb8b99-6cc9-460a-ab0f-f0eb03a3519c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:36.764618 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-80cb8b99-6cc9-460a-ab0f-f0eb03a3519c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:36.776026 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1388/2780] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:36.780547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaacb64-042b-4d5b-b619-9909b188feaa None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:36.791937 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaacb64-042b-4d5b-b619-9909b188feaa None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:36.798451 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512eaeb-8d52-4f46-9828-1941e319b28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:36.799060 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512eaeb-8d52-4f46-9828-1941e319b28f admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:36.799335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512eaeb-8d52-4f46-9828-1941e319b28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:36.799603 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512eaeb-8d52-4f46-9828-1941e319b28f admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:36.800293 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512eaeb-8d52-4f46-9828-1941e319b28f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:36.800593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512eaeb-8d52-4f46-9828-1941e319b28f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:36.802204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512eaeb-8d52-4f46-9828-1941e319b28f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:36.814211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1393/2781] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:58:36.818760 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb8b99-6cc9-460a-ab0f-f0eb03a3519c None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:36.828304 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb8b99-6cc9-460a-ab0f-f0eb03a3519c None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:36.833850 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2870f4-76c8-4cac-8b18-3f55767b5d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:36.834456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2870f4-76c8-4cac-8b18-3f55767b5d93 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:36.834732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2870f4-76c8-4cac-8b18-3f55767b5d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:36.835005 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2870f4-76c8-4cac-8b18-3f55767b5d93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:36.835660 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2870f4-76c8-4cac-8b18-3f55767b5d93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:36.837582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2870f4-76c8-4cac-8b18-3f55767b5d93 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:36.868006 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1389/2782] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:36 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:36.873733 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512eaeb-8d52-4f46-9828-1941e319b28f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:36.884099 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512eaeb-8d52-4f46-9828-1941e319b28f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:36.946375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a43706-7992-477a-93e1-59e9b654f02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:36.946926 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a43706-7992-477a-93e1-59e9b654f02c admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:36.947190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a43706-7992-477a-93e1-59e9b654f02c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:36.947450 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a43706-7992-477a-93e1-59e9b654f02c admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:36.948115 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a43706-7992-477a-93e1-59e9b654f02c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:36.950094 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a43706-7992-477a-93e1-59e9b654f02c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:36.977092 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1394/2783] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:36 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:36.978443 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2870f4-76c8-4cac-8b18-3f55767b5d93 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:36.988942 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2870f4-76c8-4cac-8b18-3f55767b5d93 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:36.993871 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d3d79-6adb-4a41-a91c-ee000d48a627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:36.994467 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d3d79-6adb-4a41-a91c-ee000d48a627 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:36.994739 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d3d79-6adb-4a41-a91c-ee000d48a627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:36.995001 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d3d79-6adb-4a41-a91c-ee000d48a627 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:36.995634 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d3d79-6adb-4a41-a91c-ee000d48a627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:36.997259 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d3d79-6adb-4a41-a91c-ee000d48a627 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:37.007661 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1390/2784] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:37.011855 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a43706-7992-477a-93e1-59e9b654f02c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:37.022749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a43706-7992-477a-93e1-59e9b654f02c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:37.028380 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1e132-9918-43ad-bd8d-a3093756d90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:37.028981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1e132-9918-43ad-bd8d-a3093756d90e admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:37.029254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1e132-9918-43ad-bd8d-a3093756d90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:37.029547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1e132-9918-43ad-bd8d-a3093756d90e admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:37.030161 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1e132-9918-43ad-bd8d-a3093756d90e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:37.032014 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1e132-9918-43ad-bd8d-a3093756d90e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:37.042871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1395/2785] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:37.048588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d3d79-6adb-4a41-a91c-ee000d48a627 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:37.058796 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d3d79-6adb-4a41-a91c-ee000d48a627 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:37.063856 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf87ae1-badb-48ee-8d40-97a4ed883417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:37.064534 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf87ae1-badb-48ee-8d40-97a4ed883417 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:37.064805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf87ae1-badb-48ee-8d40-97a4ed883417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:37.065067 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf87ae1-badb-48ee-8d40-97a4ed883417 admin admin] PATH_INFO: `/v3/projects/67b45df1ebe64acf9aabffe880a7c6ea/users/32f0e2ea07b44b059f46f8d7a2465416/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:37.077797 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:37.082192 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf87ae1-badb-48ee-8d40-97a4ed883417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b45df1ebe64acf9aabffe880a7c6ea, user_id=32f0e2ea07b44b059f46f8d7a2465416, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:37.086023 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf87ae1-badb-48ee-8d40-97a4ed883417 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:37.101814 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1391/2786] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:37 2022] PUT /identity/v3/projects/67b45df1ebe64acf9aabffe880a7c6ea/users/32f0e2ea07b44b059f46f8d7a2465416/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:37.114199 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1bf90-385e-4287-816e-c9c52166837f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:37.114484 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1bf90-385e-4287-816e-c9c52166837f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:37.114741 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1bf90-385e-4287-816e-c9c52166837f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:37.145014 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-41b1bf90-385e-4287-816e-c9c52166837f None None] MFA Rules not processed for user `32f0e2ea07b44b059f46f8d7a2465416`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:37.146243 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-41b1bf90-385e-4287-816e-c9c52166837f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:37.216655 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1396/2787] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:58:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:37.221651 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf87ae1-badb-48ee-8d40-97a4ed883417 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:37.288388 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf87ae1-badb-48ee-8d40-97a4ed883417 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:37.293799 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd097a4d-7fb2-49e9-a391-bfa6dac0ec11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:37.294404 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd097a4d-7fb2-49e9-a391-bfa6dac0ec11 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:37.294675 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd097a4d-7fb2-49e9-a391-bfa6dac0ec11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:37.294943 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd097a4d-7fb2-49e9-a391-bfa6dac0ec11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:37.295582 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd097a4d-7fb2-49e9-a391-bfa6dac0ec11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:37.297982 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd097a4d-7fb2-49e9-a391-bfa6dac0ec11 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:37.330170 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1392/2788] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:37 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:37.334670 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1bf90-385e-4287-816e-c9c52166837f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:37.348116 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1bf90-385e-4287-816e-c9c52166837f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:37.357131 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:37.408972 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75be3f-a763-4a8f-8ba1-b17890eec82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:37.409579 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75be3f-a763-4a8f-8ba1-b17890eec82e admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:37.409846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75be3f-a763-4a8f-8ba1-b17890eec82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:37.410106 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75be3f-a763-4a8f-8ba1-b17890eec82e admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:37.410736 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75be3f-a763-4a8f-8ba1-b17890eec82e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:37.412696 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75be3f-a763-4a8f-8ba1-b17890eec82e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:37.435897 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1397/2789] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:37 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:37.440716 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd097a4d-7fb2-49e9-a391-bfa6dac0ec11 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:37.455956 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd097a4d-7fb2-49e9-a391-bfa6dac0ec11 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:37.461697 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb99bf1-2f4c-46e9-97b4-0a6ef6ce8738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:37.462296 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb99bf1-2f4c-46e9-97b4-0a6ef6ce8738 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:37.462570 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb99bf1-2f4c-46e9-97b4-0a6ef6ce8738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:37.462841 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb99bf1-2f4c-46e9-97b4-0a6ef6ce8738 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:37.463487 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb99bf1-2f4c-46e9-97b4-0a6ef6ce8738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:37.465121 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb99bf1-2f4c-46e9-97b4-0a6ef6ce8738 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:37.476971 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1393/2790] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:37.482743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75be3f-a763-4a8f-8ba1-b17890eec82e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:37.493657 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75be3f-a763-4a8f-8ba1-b17890eec82e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:37.498640 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dbc36-a7c1-44be-b7a6-5f1f0593e246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:37.499250 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dbc36-a7c1-44be-b7a6-5f1f0593e246 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:37.499693 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dbc36-a7c1-44be-b7a6-5f1f0593e246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:37.499981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dbc36-a7c1-44be-b7a6-5f1f0593e246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:37.500753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dbc36-a7c1-44be-b7a6-5f1f0593e246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:37.502353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dbc36-a7c1-44be-b7a6-5f1f0593e246 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:37.512663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1398/2791] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:37.518770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb99bf1-2f4c-46e9-97b4-0a6ef6ce8738 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:37.529077 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb99bf1-2f4c-46e9-97b4-0a6ef6ce8738 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:37.534201 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b7dd9-16ee-4769-a717-412c14bad2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:37.534854 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7dd9-16ee-4769-a717-412c14bad2dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:37.535129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7dd9-16ee-4769-a717-412c14bad2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:37.535396 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7dd9-16ee-4769-a717-412c14bad2dc admin admin] PATH_INFO: `/v3/projects/ec084ef3f4a744e2a419f64ba52518be/users/b92c4b5f946c422b88e8cf2a67489beb/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:37.552159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b7dd9-16ee-4769-a717-412c14bad2dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec084ef3f4a744e2a419f64ba52518be, user_id=b92c4b5f946c422b88e8cf2a67489beb, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:37.554783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b7dd9-16ee-4769-a717-412c14bad2dc admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:37.569600 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1394/2792] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:37 2022] PUT /identity/v3/projects/ec084ef3f4a744e2a419f64ba52518be/users/b92c4b5f946c422b88e8cf2a67489beb/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:37.580391 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fe9b8-2c92-45ab-8766-4a57e4ca44e0 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:37.580694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fe9b8-2c92-45ab-8766-4a57e4ca44e0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:37.580940 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fe9b8-2c92-45ab-8766-4a57e4ca44e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:37.611038 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-471fe9b8-2c92-45ab-8766-4a57e4ca44e0 None None] MFA Rules not processed for user `b92c4b5f946c422b88e8cf2a67489beb`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:37.612939 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-471fe9b8-2c92-45ab-8766-4a57e4ca44e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:37.685331 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1399/2793] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:58:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:37.690195 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b7dd9-16ee-4769-a717-412c14bad2dc admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:37.761189 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b7dd9-16ee-4769-a717-412c14bad2dc admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:37.768356 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3191177-22c0-440e-bfe4-685002dad126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:37.768981 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3191177-22c0-440e-bfe4-685002dad126 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:37.769255 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3191177-22c0-440e-bfe4-685002dad126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:37.769518 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3191177-22c0-440e-bfe4-685002dad126 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:37.770144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3191177-22c0-440e-bfe4-685002dad126 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:37.772074 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3191177-22c0-440e-bfe4-685002dad126 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:37.807521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1395/2794] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:37 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:37.811807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fe9b8-2c92-45ab-8766-4a57e4ca44e0 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:37.823704 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fe9b8-2c92-45ab-8766-4a57e4ca44e0 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:37.882701 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d66422-c6b2-420f-9501-8fc8f7921979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:37.883320 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d66422-c6b2-420f-9501-8fc8f7921979 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:37.883596 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d66422-c6b2-420f-9501-8fc8f7921979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:37.883865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d66422-c6b2-420f-9501-8fc8f7921979 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:37.884529 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d66422-c6b2-420f-9501-8fc8f7921979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:37.886467 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d66422-c6b2-420f-9501-8fc8f7921979 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:37.910304 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1400/2795] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:37 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:37.914564 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3191177-22c0-440e-bfe4-685002dad126 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:37.925658 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3191177-22c0-440e-bfe4-685002dad126 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:37.930708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c888b45-4ce2-4f9c-b752-ec8a80a204f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:37.931307 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c888b45-4ce2-4f9c-b752-ec8a80a204f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:37.931580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c888b45-4ce2-4f9c-b752-ec8a80a204f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:37.931847 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c888b45-4ce2-4f9c-b752-ec8a80a204f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:37.932533 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c888b45-4ce2-4f9c-b752-ec8a80a204f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:37.934123 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c888b45-4ce2-4f9c-b752-ec8a80a204f1 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:37.945248 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1396/2796] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:37.949895 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d66422-c6b2-420f-9501-8fc8f7921979 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:37.960425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d66422-c6b2-420f-9501-8fc8f7921979 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:37.965227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7d269-b193-4fb4-8e5e-8d096de92166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:37.965883 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7d269-b193-4fb4-8e5e-8d096de92166 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:37.966159 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7d269-b193-4fb4-8e5e-8d096de92166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:37.966427 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7d269-b193-4fb4-8e5e-8d096de92166 admin admin] PATH_INFO: `/v3/projects/09e5a29ad9734a04acfbb1ee81ceec17/users/0664cc153b72441086ded4b9d32ce15c/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:37.969727 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:37.983454 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7d269-b193-4fb4-8e5e-8d096de92166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09e5a29ad9734a04acfbb1ee81ceec17, user_id=0664cc153b72441086ded4b9d32ce15c, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:37.986462 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7d269-b193-4fb4-8e5e-8d096de92166 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:38.003728 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1401/2797] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:37 2022] PUT /identity/v3/projects/09e5a29ad9734a04acfbb1ee81ceec17/users/0664cc153b72441086ded4b9d32ce15c/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:38.009414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c888b45-4ce2-4f9c-b752-ec8a80a204f1 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:38.020179 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c888b45-4ce2-4f9c-b752-ec8a80a204f1 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:38.032897 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:38.083951 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347746b-390d-458c-b4aa-1a28250535de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:38.084592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347746b-390d-458c-b4aa-1a28250535de admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:38.084866 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347746b-390d-458c-b4aa-1a28250535de admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:38.085130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347746b-390d-458c-b4aa-1a28250535de admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:38.085781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347746b-390d-458c-b4aa-1a28250535de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:38.087385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347746b-390d-458c-b4aa-1a28250535de admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:38.098756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1397/2798] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:38.103466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7d269-b193-4fb4-8e5e-8d096de92166 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:38.114824 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7d269-b193-4fb4-8e5e-8d096de92166 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:38.119444 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc418f-e538-443f-92a1-5cd6b1a5fd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:38.120142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc418f-e538-443f-92a1-5cd6b1a5fd10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:38.120421 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc418f-e538-443f-92a1-5cd6b1a5fd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:38.120694 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc418f-e538-443f-92a1-5cd6b1a5fd10 admin admin] PATH_INFO: `/v3/projects/09e5a29ad9734a04acfbb1ee81ceec17/users/0664cc153b72441086ded4b9d32ce15c/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:38.122585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc418f-e538-443f-92a1-5cd6b1a5fd10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09e5a29ad9734a04acfbb1ee81ceec17, user_id=0664cc153b72441086ded4b9d32ce15c, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:38.125271 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc418f-e538-443f-92a1-5cd6b1a5fd10 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:38.136524 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1402/2799] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:38 2022] PUT /identity/v3/projects/09e5a29ad9734a04acfbb1ee81ceec17/users/0664cc153b72441086ded4b9d32ce15c/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:38.148960 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c541e-1eb6-4178-a8f1-27dc04d8cb33 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:38.149339 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c541e-1eb6-4178-a8f1-27dc04d8cb33 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:38.149841 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c541e-1eb6-4178-a8f1-27dc04d8cb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:38.189712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-530c541e-1eb6-4178-a8f1-27dc04d8cb33 None None] MFA Rules not processed for user `0664cc153b72441086ded4b9d32ce15c`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:38.190972 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-530c541e-1eb6-4178-a8f1-27dc04d8cb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:38.265281 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1398/2800] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:58:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:38.276932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc418f-e538-443f-92a1-5cd6b1a5fd10 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:38.278892 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-12dc418f-e538-443f-92a1-5cd6b1a5fd10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:38.341489 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc418f-e538-443f-92a1-5cd6b1a5fd10 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:38.346815 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8f91f-4a9f-478c-82fa-99f0df6ace25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:38.347477 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8f91f-4a9f-478c-82fa-99f0df6ace25 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:38.347752 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8f91f-4a9f-478c-82fa-99f0df6ace25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:38.348018 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8f91f-4a9f-478c-82fa-99f0df6ace25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:38.357946 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8f91f-4a9f-478c-82fa-99f0df6ace25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:38.359488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8f91f-4a9f-478c-82fa-99f0df6ace25 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:38.370241 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1403/2801] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:38.932219 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c541e-1eb6-4178-a8f1-27dc04d8cb33 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:38.933853 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-530c541e-1eb6-4178-a8f1-27dc04d8cb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:38.946520 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c541e-1eb6-4178-a8f1-27dc04d8cb33 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:38.950578 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc160e9-b81c-4782-88b6-1152438ee791 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:38.951180 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc160e9-b81c-4782-88b6-1152438ee791 service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:38.951484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc160e9-b81c-4782-88b6-1152438ee791 service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:38.951757 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc160e9-b81c-4782-88b6-1152438ee791 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:39.012915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc160e9-b81c-4782-88b6-1152438ee791 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:39.014326 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc160e9-b81c-4782-88b6-1152438ee791 service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:39.024167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1399/2802] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:43.163576 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8f91f-4a9f-478c-82fa-99f0df6ace25 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:43.176560 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8f91f-4a9f-478c-82fa-99f0df6ace25 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:43.181118 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed340e9e-9b30-458a-b39c-d826ae990ffc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:43.182303 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed340e9e-9b30-458a-b39c-d826ae990ffc service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:43.182593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed340e9e-9b30-458a-b39c-d826ae990ffc service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:43.182890 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed340e9e-9b30-458a-b39c-d826ae990ffc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:43.250716 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed340e9e-9b30-458a-b39c-d826ae990ffc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:43.252200 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed340e9e-9b30-458a-b39c-d826ae990ffc service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:43.262035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1404/2803] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:45.474524 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc160e9-b81c-4782-88b6-1152438ee791 service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:45.521169 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:45.550569 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc160e9-b81c-4782-88b6-1152438ee791 service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:45.555596 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d0441a-ddde-42cb-b9d4-e1465354a827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:45.556260 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0441a-ddde-42cb-b9d4-e1465354a827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:45.556537 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0441a-ddde-42cb-b9d4-e1465354a827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:45.556811 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0441a-ddde-42cb-b9d4-e1465354a827 admin admin] PATH_INFO: `/v3/users/32f0e2ea07b44b059f46f8d7a2465416` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:45.557976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d0441a-ddde-42cb-b9d4-e1465354a827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32f0e2ea07b44b059f46f8d7a2465416)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:45.559880 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d0441a-ddde-42cb-b9d4-e1465354a827 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:45.610392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-94d0441a-ddde-42cb-b9d4-e1465354a827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32f0e2ea07b44b059f46f8d7a2465416'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:45.611141 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-94d0441a-ddde-42cb-b9d4-e1465354a827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32f0e2ea07b44b059f46f8d7a2465416'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:45.611683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-94d0441a-ddde-42cb-b9d4-e1465354a827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32f0e2ea07b44b059f46f8d7a2465416'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:45.620506 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-94d0441a-ddde-42cb-b9d4-e1465354a827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32f0e2ea07b44b059f46f8d7a2465416'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:45.633983 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-94d0441a-ddde-42cb-b9d4-e1465354a827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32f0e2ea07b44b059f46f8d7a2465416'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:45.646423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1400/2804] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:45 2022] DELETE /identity/v3/users/32f0e2ea07b44b059f46f8d7a2465416 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:45.651053 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60859363-89d5-43a7-b3c3-51f9bb6d4397 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:45.651523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60859363-89d5-43a7-b3c3-51f9bb6d4397 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:45.651627 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60859363-89d5-43a7-b3c3-51f9bb6d4397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:45.672762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-60859363-89d5-43a7-b3c3-51f9bb6d4397 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:45.673965 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-60859363-89d5-43a7-b3c3-51f9bb6d4397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:45.734454 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1405/2805] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:58:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:45.744805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d0441a-ddde-42cb-b9d4-e1465354a827 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:45.746489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-94d0441a-ddde-42cb-b9d4-e1465354a827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:45.813799 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d0441a-ddde-42cb-b9d4-e1465354a827 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:45.818393 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5af1c3-3db8-405a-b5d3-564434299e3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:45.819007 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5af1c3-3db8-405a-b5d3-564434299e3b service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:45.819286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5af1c3-3db8-405a-b5d3-564434299e3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:45.819521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5af1c3-3db8-405a-b5d3-564434299e3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:45.829312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5af1c3-3db8-405a-b5d3-564434299e3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:45.830745 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5af1c3-3db8-405a-b5d3-564434299e3b service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:45.840577 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1401/2806] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:58:46.139633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60859363-89d5-43a7-b3c3-51f9bb6d4397 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:46.141146 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-60859363-89d5-43a7-b3c3-51f9bb6d4397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:46.153912 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60859363-89d5-43a7-b3c3-51f9bb6d4397 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:46.159440 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:46.161414 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:46.161868 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:46.162159 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8 admin admin] PATH_INFO: `/v3/projects/67b45df1ebe64acf9aabffe880a7c6ea` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:46.163496 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b45df1ebe64acf9aabffe880a7c6ea)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:46.166084 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:46.175581 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:46.225703 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b45df1ebe64acf9aabffe880a7c6ea', 'request_id': 'req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.234936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b45df1ebe64acf9aabffe880a7c6ea', 'request_id': 'req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.240999 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b45df1ebe64acf9aabffe880a7c6ea', 'request_id': 'req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.257169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8 admin admin] The token cache is being invalidate because project 67b45df1ebe64acf9aabffe880a7c6ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:46.257579 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.258691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.260416 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1406/2807] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:46 2022] DELETE /identity/v3/projects/67b45df1ebe64acf9aabffe880a7c6ea => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:46.265267 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5af1c3-3db8-405a-b5d3-564434299e3b service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:46.276935 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5af1c3-3db8-405a-b5d3-564434299e3b service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:46.338995 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:46.347394 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc77944-ce8e-4919-a60d-fa37c8012c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:46.348272 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc77944-ce8e-4919-a60d-fa37c8012c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:46.348715 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc77944-ce8e-4919-a60d-fa37c8012c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:46.349400 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc77944-ce8e-4919-a60d-fa37c8012c0c admin admin] PATH_INFO: `/v3/users/b92c4b5f946c422b88e8cf2a67489beb` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:46.350816 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc77944-ce8e-4919-a60d-fa37c8012c0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b92c4b5f946c422b88e8cf2a67489beb)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:46.353130 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc77944-ce8e-4919-a60d-fa37c8012c0c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:46.410898 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-abc77944-ce8e-4919-a60d-fa37c8012c0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b92c4b5f946c422b88e8cf2a67489beb'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.411627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-abc77944-ce8e-4919-a60d-fa37c8012c0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b92c4b5f946c422b88e8cf2a67489beb'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.412288 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-abc77944-ce8e-4919-a60d-fa37c8012c0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b92c4b5f946c422b88e8cf2a67489beb'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.420852 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-abc77944-ce8e-4919-a60d-fa37c8012c0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b92c4b5f946c422b88e8cf2a67489beb'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.434427 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-abc77944-ce8e-4919-a60d-fa37c8012c0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b92c4b5f946c422b88e8cf2a67489beb'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.449041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1402/2808] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:46 2022] DELETE /identity/v3/users/b92c4b5f946c422b88e8cf2a67489beb => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:47.039271 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:47.041498 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:47.059567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e9df2-0b4d-4360-98ce-2cfa1ba6fab8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:47.120333 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:47.120975 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:47.121246 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:47.121513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423 admin admin] PATH_INFO: `/v3/projects/ec084ef3f4a744e2a419f64ba52518be` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:47.122735 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec084ef3f4a744e2a419f64ba52518be)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:47.124672 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:47.174658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec084ef3f4a744e2a419f64ba52518be', 'request_id': 'req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.183549 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec084ef3f4a744e2a419f64ba52518be', 'request_id': 'req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.189173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec084ef3f4a744e2a419f64ba52518be', 'request_id': 'req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.201314 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423 admin admin] The token cache is being invalidate because project ec084ef3f4a744e2a419f64ba52518be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:47.201597 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.202377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.204204 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1407/2809] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:47 2022] DELETE /identity/v3/projects/ec084ef3f4a744e2a419f64ba52518be => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:47.208558 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc77944-ce8e-4919-a60d-fa37c8012c0c admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:47.210198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-abc77944-ce8e-4919-a60d-fa37c8012c0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:47.224386 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc77944-ce8e-4919-a60d-fa37c8012c0c admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:47.287322 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d21787-aa45-4974-95a4-56abc739bc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:47.288914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d21787-aa45-4974-95a4-56abc739bc62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:47.289233 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d21787-aa45-4974-95a4-56abc739bc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:47.289636 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d21787-aa45-4974-95a4-56abc739bc62 admin admin] PATH_INFO: `/v3/users/0664cc153b72441086ded4b9d32ce15c` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:47.290805 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d21787-aa45-4974-95a4-56abc739bc62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0664cc153b72441086ded4b9d32ce15c)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:47.292781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d21787-aa45-4974-95a4-56abc739bc62 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:47.344041 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b7d21787-aa45-4974-95a4-56abc739bc62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0664cc153b72441086ded4b9d32ce15c'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.344856 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b7d21787-aa45-4974-95a4-56abc739bc62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0664cc153b72441086ded4b9d32ce15c'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.345494 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b7d21787-aa45-4974-95a4-56abc739bc62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0664cc153b72441086ded4b9d32ce15c'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.354853 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b7d21787-aa45-4974-95a4-56abc739bc62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0664cc153b72441086ded4b9d32ce15c'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.367671 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-b7d21787-aa45-4974-95a4-56abc739bc62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0664cc153b72441086ded4b9d32ce15c'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.373000 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:47.382414 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1403/2810] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:47 2022] DELETE /identity/v3/users/0664cc153b72441086ded4b9d32ce15c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:47.675261 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:47.676980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:47.689802 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4c6b2-5f5e-45d3-85dd-e4f57ea69423 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:47.715041 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:47.755395 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e151e8-8c90-4006-b504-552db31923f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:47.756043 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e151e8-8c90-4006-b504-552db31923f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:47.756329 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e151e8-8c90-4006-b504-552db31923f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:47.756625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e151e8-8c90-4006-b504-552db31923f8 admin admin] PATH_INFO: `/v3/projects/09e5a29ad9734a04acfbb1ee81ceec17` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:47.758400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e151e8-8c90-4006-b504-552db31923f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09e5a29ad9734a04acfbb1ee81ceec17)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:47.759751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e151e8-8c90-4006-b504-552db31923f8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:47.811722 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-53e151e8-8c90-4006-b504-552db31923f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09e5a29ad9734a04acfbb1ee81ceec17', 'request_id': 'req-53e151e8-8c90-4006-b504-552db31923f8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.821398 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-53e151e8-8c90-4006-b504-552db31923f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09e5a29ad9734a04acfbb1ee81ceec17', 'request_id': 'req-53e151e8-8c90-4006-b504-552db31923f8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.828314 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-53e151e8-8c90-4006-b504-552db31923f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09e5a29ad9734a04acfbb1ee81ceec17', 'request_id': 'req-53e151e8-8c90-4006-b504-552db31923f8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.839989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-53e151e8-8c90-4006-b504-552db31923f8 admin admin] The token cache is being invalidate because project 09e5a29ad9734a04acfbb1ee81ceec17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:47.840296 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-53e151e8-8c90-4006-b504-552db31923f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.840968 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-53e151e8-8c90-4006-b504-552db31923f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.842553 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1408/2811] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:47 2022] DELETE /identity/v3/projects/09e5a29ad9734a04acfbb1ee81ceec17 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:47.849219 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4477c3c-9e63-4fc4-a8ac-e5825cf78ab0 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:47.849499 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4477c3c-9e63-4fc4-a8ac-e5825cf78ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:47.849788 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4477c3c-9e63-4fc4-a8ac-e5825cf78ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:47.871158 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-a4477c3c-9e63-4fc4-a8ac-e5825cf78ab0 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:47.872392 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a4477c3c-9e63-4fc4-a8ac-e5825cf78ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:47.940376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1404/2812] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:47.949405 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a2b0-4fab-46d0-bdf5-3a77a35b1144 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:47.949688 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a2b0-4fab-46d0-bdf5-3a77a35b1144 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:47.949982 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a2b0-4fab-46d0-bdf5-3a77a35b1144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:47.971040 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-bd45a2b0-4fab-46d0-bdf5-3a77a35b1144 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:47.972308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bd45a2b0-4fab-46d0-bdf5-3a77a35b1144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:47.981813 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1409/2813] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:47.986587 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4477c3c-9e63-4fc4-a8ac-e5825cf78ab0 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:47.996752 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4477c3c-9e63-4fc4-a8ac-e5825cf78ab0 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:48.001883 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef765a6-91f4-496f-8f20-b0a68ea2d0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:48.002477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef765a6-91f4-496f-8f20-b0a68ea2d0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.002747 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef765a6-91f4-496f-8f20-b0a68ea2d0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.003037 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef765a6-91f4-496f-8f20-b0a68ea2d0e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.003700 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef765a6-91f4-496f-8f20-b0a68ea2d0e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:48.004018 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef765a6-91f4-496f-8f20-b0a68ea2d0e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:48.005766 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef765a6-91f4-496f-8f20-b0a68ea2d0e1 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.017773 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1405/2814] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:58:48.022347 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45a2b0-4fab-46d0-bdf5-3a77a35b1144 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:48.032019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45a2b0-4fab-46d0-bdf5-3a77a35b1144 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:48.037454 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c29046-0d71-42c3-bda2-8b70d16df3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:48.038051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c29046-0d71-42c3-bda2-8b70d16df3de admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.038321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c29046-0d71-42c3-bda2-8b70d16df3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.038696 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c29046-0d71-42c3-bda2-8b70d16df3de admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.039332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c29046-0d71-42c3-bda2-8b70d16df3de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:48.041255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c29046-0d71-42c3-bda2-8b70d16df3de admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.072747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1410/2815] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:48 2022] POST /identity/v3/projects => generated 407 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:48.077348 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef765a6-91f4-496f-8f20-b0a68ea2d0e1 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:48.092421 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef765a6-91f4-496f-8f20-b0a68ea2d0e1 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:48.151364 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce33163-0050-452c-bcd5-fc3eedb6abf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:48.151980 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce33163-0050-452c-bcd5-fc3eedb6abf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.152282 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce33163-0050-452c-bcd5-fc3eedb6abf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.152553 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce33163-0050-452c-bcd5-fc3eedb6abf5 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.153190 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce33163-0050-452c-bcd5-fc3eedb6abf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:48.155076 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce33163-0050-452c-bcd5-fc3eedb6abf5 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.178014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1406/2816] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:48 2022] POST /identity/v3/users => generated 304 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:48.182523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c29046-0d71-42c3-bda2-8b70d16df3de admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:48.194532 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c29046-0d71-42c3-bda2-8b70d16df3de admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:48.199067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91576ac8-dc40-482f-9571-af244fdb3411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:48.199667 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91576ac8-dc40-482f-9571-af244fdb3411 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.199938 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91576ac8-dc40-482f-9571-af244fdb3411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.200255 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91576ac8-dc40-482f-9571-af244fdb3411 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.200895 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91576ac8-dc40-482f-9571-af244fdb3411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:48.202487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91576ac8-dc40-482f-9571-af244fdb3411 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.212517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1411/2817] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:48.216609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce33163-0050-452c-bcd5-fc3eedb6abf5 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:48.227555 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce33163-0050-452c-bcd5-fc3eedb6abf5 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:48.233168 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f96abb-87ce-48c7-b5a9-f448f61d9bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:48.233779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f96abb-87ce-48c7-b5a9-f448f61d9bdf admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.234061 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f96abb-87ce-48c7-b5a9-f448f61d9bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.234324 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f96abb-87ce-48c7-b5a9-f448f61d9bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.234966 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f96abb-87ce-48c7-b5a9-f448f61d9bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:48.236618 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f96abb-87ce-48c7-b5a9-f448f61d9bdf admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.248212 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1407/2818] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:48.252522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91576ac8-dc40-482f-9571-af244fdb3411 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:48.263403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91576ac8-dc40-482f-9571-af244fdb3411 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:48.269063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30211e9f-3af6-4fba-8503-445a0811d464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:48.269728 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30211e9f-3af6-4fba-8503-445a0811d464 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.270117 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30211e9f-3af6-4fba-8503-445a0811d464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.270456 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30211e9f-3af6-4fba-8503-445a0811d464 admin admin] PATH_INFO: `/v3/projects/fb60c910a1654cb880b5ab26756f8140/users/2cd1713191f94e69a2e200e0f1b589d1/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.285714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30211e9f-3af6-4fba-8503-445a0811d464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb60c910a1654cb880b5ab26756f8140, user_id=2cd1713191f94e69a2e200e0f1b589d1, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:48.288619 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30211e9f-3af6-4fba-8503-445a0811d464 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.303976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1412/2819] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:48 2022] PUT /identity/v3/projects/fb60c910a1654cb880b5ab26756f8140/users/2cd1713191f94e69a2e200e0f1b589d1/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:48.317808 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542a26f-5836-4b32-b992-9b216908cb44 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.318172 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542a26f-5836-4b32-b992-9b216908cb44 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.318289 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542a26f-5836-4b32-b992-9b216908cb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.346365 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-2542a26f-5836-4b32-b992-9b216908cb44 None None] MFA Rules not processed for user `2cd1713191f94e69a2e200e0f1b589d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:48.347580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-2542a26f-5836-4b32-b992-9b216908cb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:48.399325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:48.418048 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1408/2820] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:58:48 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:48.423166 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30211e9f-3af6-4fba-8503-445a0811d464 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:48.475576 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:48.486344 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30211e9f-3af6-4fba-8503-445a0811d464 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:48.492410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec4379-89b3-43af-bcb5-8b5183728b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:48.493023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec4379-89b3-43af-bcb5-8b5183728b9a admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.493297 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec4379-89b3-43af-bcb5-8b5183728b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.493567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec4379-89b3-43af-bcb5-8b5183728b9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.494201 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec4379-89b3-43af-bcb5-8b5183728b9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:48.496140 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec4379-89b3-43af-bcb5-8b5183728b9a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.527989 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1413/2821] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:48 2022] POST /identity/v3/projects => generated 407 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:48.532403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542a26f-5836-4b32-b992-9b216908cb44 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:48.543974 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542a26f-5836-4b32-b992-9b216908cb44 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:48.609237 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5514fd-31a1-44ee-85ac-16ae65301671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:48.609953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5514fd-31a1-44ee-85ac-16ae65301671 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.610252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5514fd-31a1-44ee-85ac-16ae65301671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.610525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5514fd-31a1-44ee-85ac-16ae65301671 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.611166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5514fd-31a1-44ee-85ac-16ae65301671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:48.613141 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5514fd-31a1-44ee-85ac-16ae65301671 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.638819 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1409/2822] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:48 2022] POST /identity/v3/users => generated 310 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:48.644266 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec4379-89b3-43af-bcb5-8b5183728b9a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:48.656182 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec4379-89b3-43af-bcb5-8b5183728b9a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:48.662231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd6358-bfb5-4039-88cf-6ef072d96e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:48.662837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd6358-bfb5-4039-88cf-6ef072d96e9e admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.663109 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd6358-bfb5-4039-88cf-6ef072d96e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.663402 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd6358-bfb5-4039-88cf-6ef072d96e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.664070 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd6358-bfb5-4039-88cf-6ef072d96e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:48.665673 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd6358-bfb5-4039-88cf-6ef072d96e9e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.677209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1414/2823] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:48.681603 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5514fd-31a1-44ee-85ac-16ae65301671 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:48.691839 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5514fd-31a1-44ee-85ac-16ae65301671 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:48.696826 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aca9ba-a633-4578-a1f8-6cac0a0e6a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:48.697495 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aca9ba-a633-4578-a1f8-6cac0a0e6a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.697777 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aca9ba-a633-4578-a1f8-6cac0a0e6a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.698054 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aca9ba-a633-4578-a1f8-6cac0a0e6a0c admin admin] PATH_INFO: `/v3/projects/4c78fb4b002c45fbab1def91945f3166/users/38336eb639aa40f7ab53cdbbadbb1372/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.713456 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aca9ba-a633-4578-a1f8-6cac0a0e6a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c78fb4b002c45fbab1def91945f3166, user_id=38336eb639aa40f7ab53cdbbadbb1372, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:48.716259 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aca9ba-a633-4578-a1f8-6cac0a0e6a0c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.730869 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1410/2824] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:48 2022] PUT /identity/v3/projects/4c78fb4b002c45fbab1def91945f3166/users/38336eb639aa40f7ab53cdbbadbb1372/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:48.736459 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd6358-bfb5-4039-88cf-6ef072d96e9e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:48.747445 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd6358-bfb5-4039-88cf-6ef072d96e9e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:48.804130 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4891a-36a7-4166-b287-0cd3f6ef6678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:48.804743 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4891a-36a7-4166-b287-0cd3f6ef6678 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.805016 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4891a-36a7-4166-b287-0cd3f6ef6678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.805284 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4891a-36a7-4166-b287-0cd3f6ef6678 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.805924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4891a-36a7-4166-b287-0cd3f6ef6678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:48.807507 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4891a-36a7-4166-b287-0cd3f6ef6678 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.817442 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1415/2825] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:48.822652 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aca9ba-a633-4578-a1f8-6cac0a0e6a0c admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:48.834175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aca9ba-a633-4578-a1f8-6cac0a0e6a0c admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:48.839677 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0f5fb-4964-42a3-be9d-2e5ff7dff162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:48.840381 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0f5fb-4964-42a3-be9d-2e5ff7dff162 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.840659 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0f5fb-4964-42a3-be9d-2e5ff7dff162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.840938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0f5fb-4964-42a3-be9d-2e5ff7dff162 admin admin] PATH_INFO: `/v3/projects/4c78fb4b002c45fbab1def91945f3166/users/38336eb639aa40f7ab53cdbbadbb1372/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.842889 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0f5fb-4964-42a3-be9d-2e5ff7dff162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c78fb4b002c45fbab1def91945f3166, user_id=38336eb639aa40f7ab53cdbbadbb1372, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:48.845558 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0f5fb-4964-42a3-be9d-2e5ff7dff162 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.857649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1411/2826] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:48 2022] PUT /identity/v3/projects/4c78fb4b002c45fbab1def91945f3166/users/38336eb639aa40f7ab53cdbbadbb1372/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:48.930466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da535614-3f66-4281-bb79-6528c54606d7 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.930764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da535614-3f66-4281-bb79-6528c54606d7 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.931003 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da535614-3f66-4281-bb79-6528c54606d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.965901 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-da535614-3f66-4281-bb79-6528c54606d7 None None] MFA Rules not processed for user `38336eb639aa40f7ab53cdbbadbb1372`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:48.967142 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-da535614-3f66-4281-bb79-6528c54606d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:49.058213 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1416/2827] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:58:48 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:49.069608 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0f5fb-4964-42a3-be9d-2e5ff7dff162 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:49.071681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-a0b0f5fb-4964-42a3-be9d-2e5ff7dff162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:49.138048 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0f5fb-4964-42a3-be9d-2e5ff7dff162 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:49.142867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0512af8-b6c4-47ed-a595-164160f91dd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:49.143522 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0512af8-b6c4-47ed-a595-164160f91dd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:49.143841 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0512af8-b6c4-47ed-a595-164160f91dd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:49.144129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0512af8-b6c4-47ed-a595-164160f91dd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:49.155167 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0512af8-b6c4-47ed-a595-164160f91dd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:49.156660 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0512af8-b6c4-47ed-a595-164160f91dd8 service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:49.167308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1412/2828] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:49 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:49.344314 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da535614-3f66-4281-bb79-6528c54606d7 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:49.357457 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da535614-3f66-4281-bb79-6528c54606d7 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:49.363547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95ef9d-c512-4909-8fae-163663814273 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:49.364486 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95ef9d-c512-4909-8fae-163663814273 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:49.364860 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95ef9d-c512-4909-8fae-163663814273 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:49.365297 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95ef9d-c512-4909-8fae-163663814273 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:49.431044 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:49.447547 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95ef9d-c512-4909-8fae-163663814273 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:49.449042 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95ef9d-c512-4909-8fae-163663814273 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:49.457974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1417/2829] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:49 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:57.950526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0512af8-b6c4-47ed-a595-164160f91dd8 service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:58.002212 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:58.019103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0512af8-b6c4-47ed-a595-164160f91dd8 service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:58.024263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f4b5b-8184-439f-9315-1e87649329ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:58.024888 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f4b5b-8184-439f-9315-1e87649329ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:58.025157 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f4b5b-8184-439f-9315-1e87649329ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:58.025423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f4b5b-8184-439f-9315-1e87649329ea admin admin] PATH_INFO: `/v3/users/2cd1713191f94e69a2e200e0f1b589d1` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:58.026821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f4b5b-8184-439f-9315-1e87649329ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cd1713191f94e69a2e200e0f1b589d1)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:58.028763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f4b5b-8184-439f-9315-1e87649329ea admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:58.084790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bd3f4b5b-8184-439f-9315-1e87649329ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cd1713191f94e69a2e200e0f1b589d1'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.085845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bd3f4b5b-8184-439f-9315-1e87649329ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cd1713191f94e69a2e200e0f1b589d1'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.086781 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bd3f4b5b-8184-439f-9315-1e87649329ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cd1713191f94e69a2e200e0f1b589d1'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.096388 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bd3f4b5b-8184-439f-9315-1e87649329ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cd1713191f94e69a2e200e0f1b589d1'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.108583 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-bd3f4b5b-8184-439f-9315-1e87649329ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cd1713191f94e69a2e200e0f1b589d1'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.121767 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1413/2830] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:57 2022] DELETE /identity/v3/users/2cd1713191f94e69a2e200e0f1b589d1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:58.126458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e425c9d-9fbb-4d21-97b5-5afacdcd2fdb None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:58.127004 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e425c9d-9fbb-4d21-97b5-5afacdcd2fdb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:58.127004 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e425c9d-9fbb-4d21-97b5-5afacdcd2fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:58.148784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8e425c9d-9fbb-4d21-97b5-5afacdcd2fdb None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:58.149988 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8e425c9d-9fbb-4d21-97b5-5afacdcd2fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:58.215113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1418/2831] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:58.225227 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f4b5b-8184-439f-9315-1e87649329ea admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:58.226779 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-bd3f4b5b-8184-439f-9315-1e87649329ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:58.289631 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f4b5b-8184-439f-9315-1e87649329ea admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:58.293945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72279747-6887-42c1-b249-7aa5ade0c2ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:58.294554 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72279747-6887-42c1-b249-7aa5ade0c2ee service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:58.294799 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72279747-6887-42c1-b249-7aa5ade0c2ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:58.295094 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72279747-6887-42c1-b249-7aa5ade0c2ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:58.305939 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72279747-6887-42c1-b249-7aa5ade0c2ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:58.307385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72279747-6887-42c1-b249-7aa5ade0c2ee service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:58.318790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1414/2832] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:58.624035 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e425c9d-9fbb-4d21-97b5-5afacdcd2fdb None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:58.625940 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8e425c9d-9fbb-4d21-97b5-5afacdcd2fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:58.642716 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e425c9d-9fbb-4d21-97b5-5afacdcd2fdb None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:58.649042 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:58.650259 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:58.650730 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:58.651137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e admin admin] PATH_INFO: `/v3/projects/fb60c910a1654cb880b5ab26756f8140` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:58.652468 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb60c910a1654cb880b5ab26756f8140)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:58.654576 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:58.707924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb60c910a1654cb880b5ab26756f8140', 'request_id': 'req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.717278 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb60c910a1654cb880b5ab26756f8140', 'request_id': 'req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.723556 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb60c910a1654cb880b5ab26756f8140', 'request_id': 'req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.735382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e admin admin] The token cache is being invalidate because project fb60c910a1654cb880b5ab26756f8140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:58.735575 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.736249 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.737784 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1419/2833] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:58 2022] DELETE /identity/v3/projects/fb60c910a1654cb880b5ab26756f8140 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:58.741920 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72279747-6887-42c1-b249-7aa5ade0c2ee service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:58.752308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72279747-6887-42c1-b249-7aa5ade0c2ee service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:58.811913 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e884e75-0ad8-4acc-b373-2a2f0d3d2b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:58.812577 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e884e75-0ad8-4acc-b373-2a2f0d3d2b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:58.812876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e884e75-0ad8-4acc-b373-2a2f0d3d2b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:58.813126 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e884e75-0ad8-4acc-b373-2a2f0d3d2b07 admin admin] PATH_INFO: `/v3/users/38336eb639aa40f7ab53cdbbadbb1372` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:58.814293 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e884e75-0ad8-4acc-b373-2a2f0d3d2b07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38336eb639aa40f7ab53cdbbadbb1372)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:58.816237 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e884e75-0ad8-4acc-b373-2a2f0d3d2b07 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:58.826997 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:58.875040 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6e884e75-0ad8-4acc-b373-2a2f0d3d2b07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38336eb639aa40f7ab53cdbbadbb1372'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.875985 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6e884e75-0ad8-4acc-b373-2a2f0d3d2b07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38336eb639aa40f7ab53cdbbadbb1372'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.876670 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6e884e75-0ad8-4acc-b373-2a2f0d3d2b07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38336eb639aa40f7ab53cdbbadbb1372'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.886120 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6e884e75-0ad8-4acc-b373-2a2f0d3d2b07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38336eb639aa40f7ab53cdbbadbb1372'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.899484 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-6e884e75-0ad8-4acc-b373-2a2f0d3d2b07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38336eb639aa40f7ab53cdbbadbb1372'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.913246 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1415/2834] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:58 2022] DELETE /identity/v3/users/38336eb639aa40f7ab53cdbbadbb1372 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:59.205591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:59.207193 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:59.220974 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b8360-e832-4757-a8c0-ee1eb5c1588e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:59.280117 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59acd7-ee5f-4d94-bffd-565cc3606928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:59.280667 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59acd7-ee5f-4d94-bffd-565cc3606928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.280941 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59acd7-ee5f-4d94-bffd-565cc3606928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.281205 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59acd7-ee5f-4d94-bffd-565cc3606928 admin admin] PATH_INFO: `/v3/projects/4c78fb4b002c45fbab1def91945f3166` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.282363 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59acd7-ee5f-4d94-bffd-565cc3606928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c78fb4b002c45fbab1def91945f3166)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:59.284307 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59acd7-ee5f-4d94-bffd-565cc3606928 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:59.296700 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:59.334789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0e59acd7-ee5f-4d94-bffd-565cc3606928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c78fb4b002c45fbab1def91945f3166', 'request_id': 'req-0e59acd7-ee5f-4d94-bffd-565cc3606928'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:59.342455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0e59acd7-ee5f-4d94-bffd-565cc3606928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c78fb4b002c45fbab1def91945f3166', 'request_id': 'req-0e59acd7-ee5f-4d94-bffd-565cc3606928'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:59.348466 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0e59acd7-ee5f-4d94-bffd-565cc3606928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c78fb4b002c45fbab1def91945f3166', 'request_id': 'req-0e59acd7-ee5f-4d94-bffd-565cc3606928'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:59.359749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0e59acd7-ee5f-4d94-bffd-565cc3606928 admin admin] The token cache is being invalidate because project 4c78fb4b002c45fbab1def91945f3166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:59.360062 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0e59acd7-ee5f-4d94-bffd-565cc3606928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:59.360716 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0e59acd7-ee5f-4d94-bffd-565cc3606928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:59.362246 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1420/2835] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:59 2022] DELETE /identity/v3/projects/4c78fb4b002c45fbab1def91945f3166 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:59.367650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d970c0-4318-4b38-a0b3-59f03e3b73b5 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.368194 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d970c0-4318-4b38-a0b3-59f03e3b73b5 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.368194 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d970c0-4318-4b38-a0b3-59f03e3b73b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.389887 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-51d970c0-4318-4b38-a0b3-59f03e3b73b5 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:59.391116 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-51d970c0-4318-4b38-a0b3-59f03e3b73b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:59.453859 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1416/2836] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:59.464329 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ec703-b5c4-4c79-80be-a6892d7fb9c5 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.464608 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ec703-b5c4-4c79-80be-a6892d7fb9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.464884 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ec703-b5c4-4c79-80be-a6892d7fb9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.485920 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-5f3ec703-b5c4-4c79-80be-a6892d7fb9c5 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:59.487162 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5f3ec703-b5c4-4c79-80be-a6892d7fb9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:59.495591 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1421/2837] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:59.500038 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d970c0-4318-4b38-a0b3-59f03e3b73b5 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:59.510027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d970c0-4318-4b38-a0b3-59f03e3b73b5 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:59.514801 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de3c9b-3f0e-4abd-a42c-50058055c352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:59.515409 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de3c9b-3f0e-4abd-a42c-50058055c352 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.515687 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de3c9b-3f0e-4abd-a42c-50058055c352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.516029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de3c9b-3f0e-4abd-a42c-50058055c352 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.516708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de3c9b-3f0e-4abd-a42c-50058055c352 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:59.517002 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de3c9b-3f0e-4abd-a42c-50058055c352 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:59.518584 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de3c9b-3f0e-4abd-a42c-50058055c352 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:59.530741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1417/2838] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:58:59.535437 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ec703-b5c4-4c79-80be-a6892d7fb9c5 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:59.545929 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ec703-b5c4-4c79-80be-a6892d7fb9c5 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:59.551673 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8ca77-1f57-4be0-b2d0-be444c8ebe81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:59.552307 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8ca77-1f57-4be0-b2d0-be444c8ebe81 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.552580 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8ca77-1f57-4be0-b2d0-be444c8ebe81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.552844 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8ca77-1f57-4be0-b2d0-be444c8ebe81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.553474 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd8ca77-1f57-4be0-b2d0-be444c8ebe81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:59.555372 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd8ca77-1f57-4be0-b2d0-be444c8ebe81 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:59.587407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1422/2839] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:59 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:59.592595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de3c9b-3f0e-4abd-a42c-50058055c352 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:59.606938 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de3c9b-3f0e-4abd-a42c-50058055c352 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:59.671319 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18ad52-7e19-40f1-a838-6a4849c11a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:59.671953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18ad52-7e19-40f1-a838-6a4849c11a75 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.672580 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18ad52-7e19-40f1-a838-6a4849c11a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.672872 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18ad52-7e19-40f1-a838-6a4849c11a75 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.673524 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f18ad52-7e19-40f1-a838-6a4849c11a75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:59.675419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f18ad52-7e19-40f1-a838-6a4849c11a75 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:59.701196 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1418/2840] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:59 2022] POST /identity/v3/users => generated 284 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:59.705601 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8ca77-1f57-4be0-b2d0-be444c8ebe81 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:59.716669 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8ca77-1f57-4be0-b2d0-be444c8ebe81 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:59.722319 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e7caa-7e93-401d-a022-56b2958859bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:59.722921 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e7caa-7e93-401d-a022-56b2958859bb admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.723193 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e7caa-7e93-401d-a022-56b2958859bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.723453 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e7caa-7e93-401d-a022-56b2958859bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.724110 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e7caa-7e93-401d-a022-56b2958859bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:59.725713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e7caa-7e93-401d-a022-56b2958859bb admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:59.736225 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1423/2841] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:59.740387 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18ad52-7e19-40f1-a838-6a4849c11a75 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:59.747028 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:59.751526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18ad52-7e19-40f1-a838-6a4849c11a75 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:59.760667 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404eea7f-b268-4c39-8b29-83b608dac72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:59.761412 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404eea7f-b268-4c39-8b29-83b608dac72b admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.761875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404eea7f-b268-4c39-8b29-83b608dac72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.762329 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404eea7f-b268-4c39-8b29-83b608dac72b admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.763168 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404eea7f-b268-4c39-8b29-83b608dac72b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:59.764821 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404eea7f-b268-4c39-8b29-83b608dac72b admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:59.775913 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1419/2842] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:59.780390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e7caa-7e93-401d-a022-56b2958859bb admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:59.791403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e7caa-7e93-401d-a022-56b2958859bb admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:59.796992 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2afb66-9f3d-4be4-902f-71bd863f87fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:59.798054 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2afb66-9f3d-4be4-902f-71bd863f87fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.798329 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2afb66-9f3d-4be4-902f-71bd863f87fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.798636 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2afb66-9f3d-4be4-902f-71bd863f87fe admin admin] PATH_INFO: `/v3/projects/95239fddee90478f9eda49728b8ce3d7/users/b25746090c11462d84cb8831e88991ba/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.813818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2afb66-9f3d-4be4-902f-71bd863f87fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=95239fddee90478f9eda49728b8ce3d7, user_id=b25746090c11462d84cb8831e88991ba, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:59.816557 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2afb66-9f3d-4be4-902f-71bd863f87fe admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:59.831461 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1424/2843] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:59 2022] PUT /identity/v3/projects/95239fddee90478f9eda49728b8ce3d7/users/b25746090c11462d84cb8831e88991ba/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:59.844022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821d3eb-f4d8-490d-bb88-9e1677760c68 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.844022 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821d3eb-f4d8-490d-bb88-9e1677760c68 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.844327 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821d3eb-f4d8-490d-bb88-9e1677760c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.875991 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-5821d3eb-f4d8-490d-bb88-9e1677760c68 None None] MFA Rules not processed for user `b25746090c11462d84cb8831e88991ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:59.877242 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5821d3eb-f4d8-490d-bb88-9e1677760c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:59.959954 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1420/2844] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:58:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:59.967080 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2afb66-9f3d-4be4-902f-71bd863f87fe admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:00.039081 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2afb66-9f3d-4be4-902f-71bd863f87fe admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:00.047932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d250e1-6c0c-42af-bbac-55ef3f17ef69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:00.048758 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d250e1-6c0c-42af-bbac-55ef3f17ef69 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:00.049126 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d250e1-6c0c-42af-bbac-55ef3f17ef69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:00.049506 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d250e1-6c0c-42af-bbac-55ef3f17ef69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:00.050369 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d250e1-6c0c-42af-bbac-55ef3f17ef69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:00.053332 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d250e1-6c0c-42af-bbac-55ef3f17ef69 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:00.090238 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1425/2845] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:59 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:00.094870 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821d3eb-f4d8-490d-bb88-9e1677760c68 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:00.107898 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821d3eb-f4d8-490d-bb88-9e1677760c68 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:00.176651 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134fa12-bcb1-40fd-b15a-27bc7951907e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:00.177552 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134fa12-bcb1-40fd-b15a-27bc7951907e admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:00.178004 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134fa12-bcb1-40fd-b15a-27bc7951907e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:00.178295 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134fa12-bcb1-40fd-b15a-27bc7951907e admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:00.179245 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134fa12-bcb1-40fd-b15a-27bc7951907e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:00.182327 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134fa12-bcb1-40fd-b15a-27bc7951907e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:00.209515 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1421/2846] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:59:00 2022] POST /identity/v3/users => generated 290 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:00.215026 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d250e1-6c0c-42af-bbac-55ef3f17ef69 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:00.225756 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d250e1-6c0c-42af-bbac-55ef3f17ef69 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:00.231487 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b261e3c-b420-4cf6-97f0-85bf3ec7c9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:00.232143 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b261e3c-b420-4cf6-97f0-85bf3ec7c9ba admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:00.232418 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b261e3c-b420-4cf6-97f0-85bf3ec7c9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:00.232686 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b261e3c-b420-4cf6-97f0-85bf3ec7c9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:00.233330 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b261e3c-b420-4cf6-97f0-85bf3ec7c9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:00.234968 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b261e3c-b420-4cf6-97f0-85bf3ec7c9ba admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:00.248410 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1426/2847] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:00.253155 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134fa12-bcb1-40fd-b15a-27bc7951907e admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:00.268468 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134fa12-bcb1-40fd-b15a-27bc7951907e admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:00.274526 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8bd59-cd42-47e4-83cb-e6b1d972b4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:00.275320 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8bd59-cd42-47e4-83cb-e6b1d972b4fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:00.275603 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8bd59-cd42-47e4-83cb-e6b1d972b4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:00.275876 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8bd59-cd42-47e4-83cb-e6b1d972b4fc admin admin] PATH_INFO: `/v3/projects/1c26533c7d4a411ba3e1b003f5746517/users/2f895d3bbde24c44abf28a7243a37d35/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:00.293945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c8bd59-cd42-47e4-83cb-e6b1d972b4fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c26533c7d4a411ba3e1b003f5746517, user_id=2f895d3bbde24c44abf28a7243a37d35, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:00.296660 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c8bd59-cd42-47e4-83cb-e6b1d972b4fc admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:00.313566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1422/2848] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:00 2022] PUT /identity/v3/projects/1c26533c7d4a411ba3e1b003f5746517/users/2f895d3bbde24c44abf28a7243a37d35/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:00.317620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b261e3c-b420-4cf6-97f0-85bf3ec7c9ba admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:00.329168 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b261e3c-b420-4cf6-97f0-85bf3ec7c9ba admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:00.338005 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:00.393203 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85950809-67ab-4867-bc2e-05c3f9a99171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:00.393820 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85950809-67ab-4867-bc2e-05c3f9a99171 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:00.394101 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85950809-67ab-4867-bc2e-05c3f9a99171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:00.394368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85950809-67ab-4867-bc2e-05c3f9a99171 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:00.395024 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85950809-67ab-4867-bc2e-05c3f9a99171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:00.396700 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85950809-67ab-4867-bc2e-05c3f9a99171 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:00.407058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1427/2849] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:00.411645 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8bd59-cd42-47e4-83cb-e6b1d972b4fc admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:00.423144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8bd59-cd42-47e4-83cb-e6b1d972b4fc admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:00.428949 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb15d8-e7a6-45be-b85c-15ce0f4d199a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:00.429915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb15d8-e7a6-45be-b85c-15ce0f4d199a admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:00.430318 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb15d8-e7a6-45be-b85c-15ce0f4d199a admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:00.430711 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb15d8-e7a6-45be-b85c-15ce0f4d199a admin admin] PATH_INFO: `/v3/projects/1c26533c7d4a411ba3e1b003f5746517/users/2f895d3bbde24c44abf28a7243a37d35/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:00.433937 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb15d8-e7a6-45be-b85c-15ce0f4d199a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c26533c7d4a411ba3e1b003f5746517, user_id=2f895d3bbde24c44abf28a7243a37d35, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:00.436625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb15d8-e7a6-45be-b85c-15ce0f4d199a admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:00.448093 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1423/2850] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:00 2022] PUT /identity/v3/projects/1c26533c7d4a411ba3e1b003f5746517/users/2f895d3bbde24c44abf28a7243a37d35/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:00.458948 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e301d25-3741-489e-b5f7-85db14217884 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:00.459231 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e301d25-3741-489e-b5f7-85db14217884 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:00.459504 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e301d25-3741-489e-b5f7-85db14217884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:00.490403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9e301d25-3741-489e-b5f7-85db14217884 None None] MFA Rules not processed for user `2f895d3bbde24c44abf28a7243a37d35`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:00.491629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9e301d25-3741-489e-b5f7-85db14217884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:00.570700 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1428/2851] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:00.583685 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb15d8-e7a6-45be-b85c-15ce0f4d199a admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:00.585790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-d9eb15d8-e7a6-45be-b85c-15ce0f4d199a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:00.627836 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:00.648947 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb15d8-e7a6-45be-b85c-15ce0f4d199a admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:00.653110 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebc533-da5c-495c-91da-903bfc57d96d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:00.653719 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebc533-da5c-495c-91da-903bfc57d96d service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:00.653992 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebc533-da5c-495c-91da-903bfc57d96d service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:00.654258 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebc533-da5c-495c-91da-903bfc57d96d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:00.667385 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebc533-da5c-495c-91da-903bfc57d96d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:00.668833 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebc533-da5c-495c-91da-903bfc57d96d service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:00.681279 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1424/2852] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:00.774962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e301d25-3741-489e-b5f7-85db14217884 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:00.786202 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e301d25-3741-489e-b5f7-85db14217884 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:00.790527 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310d6eb-e77b-4f55-9f66-dab5c1d85c0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:00.791192 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310d6eb-e77b-4f55-9f66-dab5c1d85c0e service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:00.791564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310d6eb-e77b-4f55-9f66-dab5c1d85c0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:00.791962 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310d6eb-e77b-4f55-9f66-dab5c1d85c0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:00.850556 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c310d6eb-e77b-4f55-9f66-dab5c1d85c0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:00.851927 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c310d6eb-e77b-4f55-9f66-dab5c1d85c0e service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:00.861909 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1429/2853] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:05.129525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebc533-da5c-495c-91da-903bfc57d96d service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:05.131284 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-b4ebc533-da5c-495c-91da-903bfc57d96d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:05.196757 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebc533-da5c-495c-91da-903bfc57d96d service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:05.201627 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2dcb1-4460-4cc9-a1bc-bf0114d1b7c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:05.202235 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2dcb1-4460-4cc9-a1bc-bf0114d1b7c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:05.202509 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2dcb1-4460-4cc9-a1bc-bf0114d1b7c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:05.202772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2dcb1-4460-4cc9-a1bc-bf0114d1b7c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:05.204801 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-21c2dcb1-4460-4cc9-a1bc-bf0114d1b7c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:05.282703 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2dcb1-4460-4cc9-a1bc-bf0114d1b7c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:05.284070 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2dcb1-4460-4cc9-a1bc-bf0114d1b7c1 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:05.296374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1425/2854] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:05.913412 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310d6eb-e77b-4f55-9f66-dab5c1d85c0e service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:05.953075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:05.995299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310d6eb-e77b-4f55-9f66-dab5c1d85c0e service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:06.002085 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff8087-6be6-401a-aa2f-61057dc7b9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:06.002727 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff8087-6be6-401a-aa2f-61057dc7b9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:06.002994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff8087-6be6-401a-aa2f-61057dc7b9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:06.003249 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff8087-6be6-401a-aa2f-61057dc7b9aa admin admin] PATH_INFO: `/v3/users/b25746090c11462d84cb8831e88991ba` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:06.005260 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff8087-6be6-401a-aa2f-61057dc7b9aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b25746090c11462d84cb8831e88991ba)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:06.008685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff8087-6be6-401a-aa2f-61057dc7b9aa admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:06.062190 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0eff8087-6be6-401a-aa2f-61057dc7b9aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b25746090c11462d84cb8831e88991ba'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:06.069282 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0eff8087-6be6-401a-aa2f-61057dc7b9aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b25746090c11462d84cb8831e88991ba'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:06.077239 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0eff8087-6be6-401a-aa2f-61057dc7b9aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b25746090c11462d84cb8831e88991ba'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:06.078051 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0eff8087-6be6-401a-aa2f-61057dc7b9aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b25746090c11462d84cb8831e88991ba'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:06.090508 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0eff8087-6be6-401a-aa2f-61057dc7b9aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b25746090c11462d84cb8831e88991ba'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:06.097014 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1430/2855] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:59:05 2022] DELETE /identity/v3/users/b25746090c11462d84cb8831e88991ba => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:06.101653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703ff2b-b572-4220-a612-1872b787c015 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:06.102098 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703ff2b-b572-4220-a612-1872b787c015 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:06.102373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703ff2b-b572-4220-a612-1872b787c015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:06.124350 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-e703ff2b-b572-4220-a612-1872b787c015 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:06.125581 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e703ff2b-b572-4220-a612-1872b787c015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:06.196993 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1426/2856] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:06.208821 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff8087-6be6-401a-aa2f-61057dc7b9aa admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:06.210732 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0eff8087-6be6-401a-aa2f-61057dc7b9aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:06.275706 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff8087-6be6-401a-aa2f-61057dc7b9aa admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:06.281670 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4035c-a536-4c4a-a2a3-27c2f7c5714b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:06.282288 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4035c-a536-4c4a-a2a3-27c2f7c5714b service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:06.282567 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4035c-a536-4c4a-a2a3-27c2f7c5714b service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:06.282833 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4035c-a536-4c4a-a2a3-27c2f7c5714b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:06.294067 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4035c-a536-4c4a-a2a3-27c2f7c5714b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:06.295521 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4035c-a536-4c4a-a2a3-27c2f7c5714b service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:06.306690 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1431/2857] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:06.604661 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703ff2b-b572-4220-a612-1872b787c015 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:06.606620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e703ff2b-b572-4220-a612-1872b787c015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:06.628321 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703ff2b-b572-4220-a612-1872b787c015 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:06.632894 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806cbffc-3c3b-4c13-8146-7875ce582dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:06.633621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cbffc-3c3b-4c13-8146-7875ce582dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:06.633962 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cbffc-3c3b-4c13-8146-7875ce582dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:06.634295 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cbffc-3c3b-4c13-8146-7875ce582dc9 admin admin] PATH_INFO: `/v3/projects/95239fddee90478f9eda49728b8ce3d7` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:06.635688 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806cbffc-3c3b-4c13-8146-7875ce582dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95239fddee90478f9eda49728b8ce3d7)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:06.638132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806cbffc-3c3b-4c13-8146-7875ce582dc9 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:06.656573 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=100 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:06.692341 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-806cbffc-3c3b-4c13-8146-7875ce582dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95239fddee90478f9eda49728b8ce3d7', 'request_id': 'req-806cbffc-3c3b-4c13-8146-7875ce582dc9'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:06.697709 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-806cbffc-3c3b-4c13-8146-7875ce582dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95239fddee90478f9eda49728b8ce3d7', 'request_id': 'req-806cbffc-3c3b-4c13-8146-7875ce582dc9'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:06.704446 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-806cbffc-3c3b-4c13-8146-7875ce582dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95239fddee90478f9eda49728b8ce3d7', 'request_id': 'req-806cbffc-3c3b-4c13-8146-7875ce582dc9'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:06.718800 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-806cbffc-3c3b-4c13-8146-7875ce582dc9 admin admin] The token cache is being invalidate because project 95239fddee90478f9eda49728b8ce3d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:59:06.719017 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-806cbffc-3c3b-4c13-8146-7875ce582dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:06.719643 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-806cbffc-3c3b-4c13-8146-7875ce582dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:06.721221 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1427/2858] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:59:06 2022] DELETE /identity/v3/projects/95239fddee90478f9eda49728b8ce3d7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:06.725881 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4035c-a536-4c4a-a2a3-27c2f7c5714b service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:06.736282 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4035c-a536-4c4a-a2a3-27c2f7c5714b service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:06.787851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:06.799295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65aaf67-50f2-4153-9948-28ef8fc9a129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:06.799960 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65aaf67-50f2-4153-9948-28ef8fc9a129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:06.800316 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65aaf67-50f2-4153-9948-28ef8fc9a129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:06.800595 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65aaf67-50f2-4153-9948-28ef8fc9a129 admin admin] PATH_INFO: `/v3/users/2f895d3bbde24c44abf28a7243a37d35` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:06.801865 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65aaf67-50f2-4153-9948-28ef8fc9a129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f895d3bbde24c44abf28a7243a37d35)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:06.803980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65aaf67-50f2-4153-9948-28ef8fc9a129 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:06.854607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d65aaf67-50f2-4153-9948-28ef8fc9a129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f895d3bbde24c44abf28a7243a37d35'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:06.861837 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d65aaf67-50f2-4153-9948-28ef8fc9a129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f895d3bbde24c44abf28a7243a37d35'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:06.873892 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d65aaf67-50f2-4153-9948-28ef8fc9a129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f895d3bbde24c44abf28a7243a37d35'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:06.874965 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d65aaf67-50f2-4153-9948-28ef8fc9a129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f895d3bbde24c44abf28a7243a37d35'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:06.894356 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d65aaf67-50f2-4153-9948-28ef8fc9a129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f895d3bbde24c44abf28a7243a37d35'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:06.901198 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1432/2859] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:59:06 2022] DELETE /identity/v3/users/2f895d3bbde24c44abf28a7243a37d35 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:59:07.196712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806cbffc-3c3b-4c13-8146-7875ce582dc9 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:07.198824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-806cbffc-3c3b-4c13-8146-7875ce582dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:07.215069 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806cbffc-3c3b-4c13-8146-7875ce582dc9 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:07.297751 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5eafc-8050-4d10-ad9a-7c22ea07553e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:07.298309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5eafc-8050-4d10-ad9a-7c22ea07553e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:07.298642 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5eafc-8050-4d10-ad9a-7c22ea07553e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:07.298915 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5eafc-8050-4d10-ad9a-7c22ea07553e admin admin] PATH_INFO: `/v3/projects/1c26533c7d4a411ba3e1b003f5746517` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:07.300129 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5eafc-8050-4d10-ad9a-7c22ea07553e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c26533c7d4a411ba3e1b003f5746517)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:07.302054 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5eafc-8050-4d10-ad9a-7c22ea07553e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:07.350419 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-74b5eafc-8050-4d10-ad9a-7c22ea07553e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c26533c7d4a411ba3e1b003f5746517', 'request_id': 'req-74b5eafc-8050-4d10-ad9a-7c22ea07553e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:07.356120 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-74b5eafc-8050-4d10-ad9a-7c22ea07553e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c26533c7d4a411ba3e1b003f5746517', 'request_id': 'req-74b5eafc-8050-4d10-ad9a-7c22ea07553e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:07.362734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-74b5eafc-8050-4d10-ad9a-7c22ea07553e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c26533c7d4a411ba3e1b003f5746517', 'request_id': 'req-74b5eafc-8050-4d10-ad9a-7c22ea07553e'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:07.377214 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-74b5eafc-8050-4d10-ad9a-7c22ea07553e admin admin] The token cache is being invalidate because project 1c26533c7d4a411ba3e1b003f5746517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:59:07.377500 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-74b5eafc-8050-4d10-ad9a-7c22ea07553e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:07.378111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-74b5eafc-8050-4d10-ad9a-7c22ea07553e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:07.379598 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1428/2860] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:59:07 2022] DELETE /identity/v3/projects/1c26533c7d4a411ba3e1b003f5746517 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:07.384995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf58e66-14d3-459a-9932-6423921d82ea None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:07.385571 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf58e66-14d3-459a-9932-6423921d82ea None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:07.385697 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf58e66-14d3-459a-9932-6423921d82ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:07.405976 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-baf58e66-14d3-459a-9932-6423921d82ea None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:07.407295 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-baf58e66-14d3-459a-9932-6423921d82ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:07.470619 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1433/2861] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:07.478922 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d161f25-d396-46fa-b71b-bc9414d95875 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:07.479346 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d161f25-d396-46fa-b71b-bc9414d95875 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:07.479626 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d161f25-d396-46fa-b71b-bc9414d95875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:07.502592 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-7d161f25-d396-46fa-b71b-bc9414d95875 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:07.503846 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-7d161f25-d396-46fa-b71b-bc9414d95875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:07.512558 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1429/2862] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:07.516971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf58e66-14d3-459a-9932-6423921d82ea None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:07.526905 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf58e66-14d3-459a-9932-6423921d82ea None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:07.531585 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02898d4f-2403-4f41-9a53-8aef2e9f37c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:07.532218 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02898d4f-2403-4f41-9a53-8aef2e9f37c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:07.532492 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02898d4f-2403-4f41-9a53-8aef2e9f37c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:07.532761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02898d4f-2403-4f41-9a53-8aef2e9f37c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:07.533407 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02898d4f-2403-4f41-9a53-8aef2e9f37c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:59:07.533704 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02898d4f-2403-4f41-9a53-8aef2e9f37c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:07.535298 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02898d4f-2403-4f41-9a53-8aef2e9f37c7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:07.548818 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1434/2863] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:59:07.553290 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d161f25-d396-46fa-b71b-bc9414d95875 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:07.563186 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d161f25-d396-46fa-b71b-bc9414d95875 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:07.569075 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98bf7a-5cb0-4159-be8f-29fc967e2f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:07.569677 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98bf7a-5cb0-4159-be8f-29fc967e2f99 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:07.569946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98bf7a-5cb0-4159-be8f-29fc967e2f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:07.570231 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98bf7a-5cb0-4159-be8f-29fc967e2f99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:07.570832 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98bf7a-5cb0-4159-be8f-29fc967e2f99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:07.572803 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98bf7a-5cb0-4159-be8f-29fc967e2f99 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:07.606934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1430/2864] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:59:07 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:07.611789 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02898d4f-2403-4f41-9a53-8aef2e9f37c7 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:07.626663 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02898d4f-2403-4f41-9a53-8aef2e9f37c7 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:07.686759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2881f2-30e8-4bb0-8238-1d18a560da75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:07.687397 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2881f2-30e8-4bb0-8238-1d18a560da75 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:07.687703 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2881f2-30e8-4bb0-8238-1d18a560da75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:07.687975 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2881f2-30e8-4bb0-8238-1d18a560da75 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:07.688734 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2881f2-30e8-4bb0-8238-1d18a560da75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:07.690692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2881f2-30e8-4bb0-8238-1d18a560da75 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:07.699215 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:07.717213 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1435/2865] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:59:07 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:07.720070 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98bf7a-5cb0-4159-be8f-29fc967e2f99 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:07.726683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:07.733620 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98bf7a-5cb0-4159-be8f-29fc967e2f99 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:07.740427 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7491139-b538-4fa8-9ef8-40548d845763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:07.741137 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7491139-b538-4fa8-9ef8-40548d845763 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:07.741480 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7491139-b538-4fa8-9ef8-40548d845763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:07.741816 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7491139-b538-4fa8-9ef8-40548d845763 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:07.742537 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7491139-b538-4fa8-9ef8-40548d845763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:07.744465 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7491139-b538-4fa8-9ef8-40548d845763 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:07.757588 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1431/2866] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:07.759639 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2881f2-30e8-4bb0-8238-1d18a560da75 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:07.771307 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2881f2-30e8-4bb0-8238-1d18a560da75 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:07.776588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d809d-25fa-465e-8142-abe3d9d410ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:07.777315 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d809d-25fa-465e-8142-abe3d9d410ed admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:07.777600 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d809d-25fa-465e-8142-abe3d9d410ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:07.777933 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d809d-25fa-465e-8142-abe3d9d410ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:07.779302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d809d-25fa-465e-8142-abe3d9d410ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:07.781033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d809d-25fa-465e-8142-abe3d9d410ed admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:07.792518 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1436/2867] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:07.797431 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7491139-b538-4fa8-9ef8-40548d845763 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:07.807286 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7491139-b538-4fa8-9ef8-40548d845763 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:07.812684 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830cc21-201d-474d-b7a8-a886c983af32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:07.813344 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830cc21-201d-474d-b7a8-a886c983af32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:07.813619 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830cc21-201d-474d-b7a8-a886c983af32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:07.813889 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830cc21-201d-474d-b7a8-a886c983af32 admin admin] PATH_INFO: `/v3/projects/b3b2e41fde944cd3b16b9817e24a861c/users/b4b92fbe10f74e548fb54fafe00f2dac/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:07.835241 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830cc21-201d-474d-b7a8-a886c983af32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3b2e41fde944cd3b16b9817e24a861c, user_id=b4b92fbe10f74e548fb54fafe00f2dac, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:07.839641 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830cc21-201d-474d-b7a8-a886c983af32 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:07.855927 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1432/2868] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:07 2022] PUT /identity/v3/projects/b3b2e41fde944cd3b16b9817e24a861c/users/b4b92fbe10f74e548fb54fafe00f2dac/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:07.868368 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07403b5-ebba-4073-983b-ee75ddade39b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:07.868607 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07403b5-ebba-4073-983b-ee75ddade39b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:07.868909 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07403b5-ebba-4073-983b-ee75ddade39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:07.898676 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c07403b5-ebba-4073-983b-ee75ddade39b None None] MFA Rules not processed for user `b4b92fbe10f74e548fb54fafe00f2dac`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:07.899971 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c07403b5-ebba-4073-983b-ee75ddade39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:07.969493 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1437/2869] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:07.974289 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830cc21-201d-474d-b7a8-a886c983af32 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:08.051946 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830cc21-201d-474d-b7a8-a886c983af32 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:08.057770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22ea49-dc6b-44ca-b3d4-c7fe60985241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:08.058674 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22ea49-dc6b-44ca-b3d4-c7fe60985241 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:08.059131 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22ea49-dc6b-44ca-b3d4-c7fe60985241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:08.059598 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22ea49-dc6b-44ca-b3d4-c7fe60985241 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:08.060623 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22ea49-dc6b-44ca-b3d4-c7fe60985241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:08.063680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22ea49-dc6b-44ca-b3d4-c7fe60985241 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:08.105244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1433/2870] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:59:07 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:08.110163 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07403b5-ebba-4073-983b-ee75ddade39b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:08.122915 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07403b5-ebba-4073-983b-ee75ddade39b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:08.199512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c314439-1442-4132-ab52-bd6ac653ccdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:08.200156 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c314439-1442-4132-ab52-bd6ac653ccdc admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:08.200431 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c314439-1442-4132-ab52-bd6ac653ccdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:08.200761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c314439-1442-4132-ab52-bd6ac653ccdc admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:08.201590 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c314439-1442-4132-ab52-bd6ac653ccdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:08.203514 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c314439-1442-4132-ab52-bd6ac653ccdc admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:08.226737 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1438/2871] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:59:08 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:08.231156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22ea49-dc6b-44ca-b3d4-c7fe60985241 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:08.242315 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22ea49-dc6b-44ca-b3d4-c7fe60985241 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:08.248138 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f137c7-cc29-427d-a3e6-bb94db0d41a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:08.248472 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f137c7-cc29-427d-a3e6-bb94db0d41a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:08.248765 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f137c7-cc29-427d-a3e6-bb94db0d41a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:08.249035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f137c7-cc29-427d-a3e6-bb94db0d41a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:08.249673 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f137c7-cc29-427d-a3e6-bb94db0d41a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:08.251566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f137c7-cc29-427d-a3e6-bb94db0d41a1 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:08.262820 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1434/2872] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:08.268185 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c314439-1442-4132-ab52-bd6ac653ccdc admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:08.282499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c314439-1442-4132-ab52-bd6ac653ccdc admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:08.290807 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54cb0f-44a3-4ca7-ab7c-3b741bdfcb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:08.291662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54cb0f-44a3-4ca7-ab7c-3b741bdfcb96 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:08.292044 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54cb0f-44a3-4ca7-ab7c-3b741bdfcb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:08.292391 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54cb0f-44a3-4ca7-ab7c-3b741bdfcb96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:08.293349 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54cb0f-44a3-4ca7-ab7c-3b741bdfcb96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:08.295648 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54cb0f-44a3-4ca7-ab7c-3b741bdfcb96 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:08.311007 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1439/2873] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:08.315429 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f137c7-cc29-427d-a3e6-bb94db0d41a1 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:08.326794 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f137c7-cc29-427d-a3e6-bb94db0d41a1 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:08.333250 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c463b-870b-40de-8258-6d7ded10dd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:08.334103 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c463b-870b-40de-8258-6d7ded10dd32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:08.334377 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c463b-870b-40de-8258-6d7ded10dd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:08.334644 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c463b-870b-40de-8258-6d7ded10dd32 admin admin] PATH_INFO: `/v3/projects/a3b45b98bb5344d8a1355b52215c8abf/users/6b1fa44d75854e1d8800bb39a36dd20d/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:08.352658 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c463b-870b-40de-8258-6d7ded10dd32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3b45b98bb5344d8a1355b52215c8abf, user_id=6b1fa44d75854e1d8800bb39a36dd20d, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:08.355288 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c463b-870b-40de-8258-6d7ded10dd32 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:08.372475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1435/2874] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:08 2022] PUT /identity/v3/projects/a3b45b98bb5344d8a1355b52215c8abf/users/6b1fa44d75854e1d8800bb39a36dd20d/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:08.382626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b5e5e-c38f-4fe9-9bee-4ab2bfe1490b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:08.382917 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b5e5e-c38f-4fe9-9bee-4ab2bfe1490b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:08.383184 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b5e5e-c38f-4fe9-9bee-4ab2bfe1490b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:08.416075 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-237b5e5e-c38f-4fe9-9bee-4ab2bfe1490b None None] MFA Rules not processed for user `6b1fa44d75854e1d8800bb39a36dd20d`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:08.417353 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-237b5e5e-c38f-4fe9-9bee-4ab2bfe1490b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:08.460871 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:08.500435 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1440/2875] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:08.507525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c463b-870b-40de-8258-6d7ded10dd32 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:08.584538 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c463b-870b-40de-8258-6d7ded10dd32 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:08.591646 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233eb8cd-1d21-4094-81ee-90f7bb1fc920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:08.592455 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233eb8cd-1d21-4094-81ee-90f7bb1fc920 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:08.592734 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233eb8cd-1d21-4094-81ee-90f7bb1fc920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:08.593002 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233eb8cd-1d21-4094-81ee-90f7bb1fc920 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:08.593664 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233eb8cd-1d21-4094-81ee-90f7bb1fc920 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:08.595597 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233eb8cd-1d21-4094-81ee-90f7bb1fc920 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:08.628661 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1436/2876] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:59:08 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:08.630898 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b5e5e-c38f-4fe9-9bee-4ab2bfe1490b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:08.643100 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b5e5e-c38f-4fe9-9bee-4ab2bfe1490b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:08.699522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff7c75-17e8-4800-a6b1-246711d3b296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:08.700170 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff7c75-17e8-4800-a6b1-246711d3b296 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:08.700455 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff7c75-17e8-4800-a6b1-246711d3b296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:08.700738 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff7c75-17e8-4800-a6b1-246711d3b296 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:08.701351 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff7c75-17e8-4800-a6b1-246711d3b296 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:08.703604 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff7c75-17e8-4800-a6b1-246711d3b296 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:08.727321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1441/2877] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:59:08 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:08.731628 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233eb8cd-1d21-4094-81ee-90f7bb1fc920 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:08.738649 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:08.742836 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233eb8cd-1d21-4094-81ee-90f7bb1fc920 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:08.748769 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2d3b5-ce7a-46ae-8f02-ba2e059f8b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:08.749381 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2d3b5-ce7a-46ae-8f02-ba2e059f8b5d admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:08.749658 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2d3b5-ce7a-46ae-8f02-ba2e059f8b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:08.749924 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2d3b5-ce7a-46ae-8f02-ba2e059f8b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:08.750566 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2d3b5-ce7a-46ae-8f02-ba2e059f8b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:08.752197 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2d3b5-ce7a-46ae-8f02-ba2e059f8b5d admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:08.763071 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1437/2878] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:08.768285 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff7c75-17e8-4800-a6b1-246711d3b296 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:08.783390 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff7c75-17e8-4800-a6b1-246711d3b296 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:08.791571 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabe879-2f9d-4108-9196-3bb485fce94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:08.791960 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabe879-2f9d-4108-9196-3bb485fce94c admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:08.791960 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabe879-2f9d-4108-9196-3bb485fce94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:08.792462 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabe879-2f9d-4108-9196-3bb485fce94c admin admin] PATH_INFO: `/v3/projects/b83329f4ecd9462091615b0b2414b74f/users/4e4a3c52ffcd4801b0eb05260f2d48d7/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:08.810518 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabe879-2f9d-4108-9196-3bb485fce94c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b83329f4ecd9462091615b0b2414b74f, user_id=4e4a3c52ffcd4801b0eb05260f2d48d7, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:08.814341 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabe879-2f9d-4108-9196-3bb485fce94c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:08.833375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1442/2879] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:08 2022] PUT /identity/v3/projects/b83329f4ecd9462091615b0b2414b74f/users/4e4a3c52ffcd4801b0eb05260f2d48d7/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:08.837244 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2d3b5-ce7a-46ae-8f02-ba2e059f8b5d admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:08.847737 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2d3b5-ce7a-46ae-8f02-ba2e059f8b5d admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:08.881374 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabe879-2f9d-4108-9196-3bb485fce94c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:08.883458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9eabe879-2f9d-4108-9196-3bb485fce94c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:08.908333 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed65c7-4681-42ae-ba3a-537ea95ce4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:08.908968 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed65c7-4681-42ae-ba3a-537ea95ce4a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:08.909236 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed65c7-4681-42ae-ba3a-537ea95ce4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:08.909497 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed65c7-4681-42ae-ba3a-537ea95ce4a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:08.910141 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed65c7-4681-42ae-ba3a-537ea95ce4a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:08.911723 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed65c7-4681-42ae-ba3a-537ea95ce4a3 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:08.921681 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1438/2880] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:08.926449 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed65c7-4681-42ae-ba3a-537ea95ce4a3 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:08.936790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed65c7-4681-42ae-ba3a-537ea95ce4a3 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:08.941433 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc78ce-b1c8-485b-85cd-355e5b401fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:08.942096 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc78ce-b1c8-485b-85cd-355e5b401fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:08.942376 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc78ce-b1c8-485b-85cd-355e5b401fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:08.942648 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc78ce-b1c8-485b-85cd-355e5b401fa8 admin admin] PATH_INFO: `/v3/projects/b83329f4ecd9462091615b0b2414b74f/users/4e4a3c52ffcd4801b0eb05260f2d48d7/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:08.944575 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc78ce-b1c8-485b-85cd-355e5b401fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b83329f4ecd9462091615b0b2414b74f, user_id=4e4a3c52ffcd4801b0eb05260f2d48d7, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:08.947202 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc78ce-b1c8-485b-85cd-355e5b401fa8 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:08.952708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabe879-2f9d-4108-9196-3bb485fce94c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:08.957929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1439/2881] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:08 2022] PUT /identity/v3/projects/b83329f4ecd9462091615b0b2414b74f/users/4e4a3c52ffcd4801b0eb05260f2d48d7/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:59:08.969465 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f8d8a-e42c-4cef-bb49-0f2f87de5948 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:08.969748 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f8d8a-e42c-4cef-bb49-0f2f87de5948 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:08.970011 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f8d8a-e42c-4cef-bb49-0f2f87de5948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:09.004933 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-f48f8d8a-e42c-4cef-bb49-0f2f87de5948 None None] MFA Rules not processed for user `4e4a3c52ffcd4801b0eb05260f2d48d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:09.006312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f48f8d8a-e42c-4cef-bb49-0f2f87de5948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:09.016499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343e2a3-aa84-4c29-9828-7d0723a60f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:09.017113 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343e2a3-aa84-4c29-9828-7d0723a60f3d service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:09.017386 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343e2a3-aa84-4c29-9828-7d0723a60f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:09.017649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343e2a3-aa84-4c29-9828-7d0723a60f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:09.019499 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2343e2a3-aa84-4c29-9828-7d0723a60f3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:09.030510 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:09.074875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1440/2882] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:59:09.086274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f8d8a-e42c-4cef-bb49-0f2f87de5948 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:09.087653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f48f8d8a-e42c-4cef-bb49-0f2f87de5948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:09.098589 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343e2a3-aa84-4c29-9828-7d0723a60f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:09.100079 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343e2a3-aa84-4c29-9828-7d0723a60f3d service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:09.112366 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1443/2883] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:09.146959 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f8d8a-e42c-4cef-bb49-0f2f87de5948 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:09.150943 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85551c9b-d8a5-445c-b187-54cc17a81fe6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:09.151824 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85551c9b-d8a5-445c-b187-54cc17a81fe6 service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:09.151977 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85551c9b-d8a5-445c-b187-54cc17a81fe6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:09.152306 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85551c9b-d8a5-445c-b187-54cc17a81fe6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:09.167732 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:09.218109 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85551c9b-d8a5-445c-b187-54cc17a81fe6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:09.219555 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85551c9b-d8a5-445c-b187-54cc17a81fe6 service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:09.229122 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1441/2884] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:59:10.962863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343e2a3-aa84-4c29-9828-7d0723a60f3d service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:10.964915 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2343e2a3-aa84-4c29-9828-7d0723a60f3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:10.982377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343e2a3-aa84-4c29-9828-7d0723a60f3d service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:10.986588 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4c489-4247-4219-95dc-9f1d713d9538 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:10.987183 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4c489-4247-4219-95dc-9f1d713d9538 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:10.987488 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4c489-4247-4219-95dc-9f1d713d9538 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:10.987754 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4c489-4247-4219-95dc-9f1d713d9538 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:11.048824 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af4c489-4247-4219-95dc-9f1d713d9538 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:11.050267 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af4c489-4247-4219-95dc-9f1d713d9538 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:11.060097 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1444/2885] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:11.662571 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85551c9b-d8a5-445c-b187-54cc17a81fe6 service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:11.673337 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85551c9b-d8a5-445c-b187-54cc17a81fe6 service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:11.677134 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de08e6-8c26-4751-9dc4-da97b684543e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:11.677733 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de08e6-8c26-4751-9dc4-da97b684543e service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:11.678010 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de08e6-8c26-4751-9dc4-da97b684543e service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:11.678279 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de08e6-8c26-4751-9dc4-da97b684543e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:11.687673 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de08e6-8c26-4751-9dc4-da97b684543e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:11.689166 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de08e6-8c26-4751-9dc4-da97b684543e service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:11.699205 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1442/2886] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:15.641137 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4c489-4247-4219-95dc-9f1d713d9538 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:15.715275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4c489-4247-4219-95dc-9f1d713d9538 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:15.720143 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9497a-2f05-4c2e-b480-450d4216f7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:15.720782 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9497a-2f05-4c2e-b480-450d4216f7a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:15.721062 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9497a-2f05-4c2e-b480-450d4216f7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:15.721331 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9497a-2f05-4c2e-b480-450d4216f7a1 admin admin] PATH_INFO: `/v3/users/b4b92fbe10f74e548fb54fafe00f2dac` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:15.722481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9497a-2f05-4c2e-b480-450d4216f7a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4b92fbe10f74e548fb54fafe00f2dac)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:15.724621 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9497a-2f05-4c2e-b480-450d4216f7a1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:15.778130 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:15.788090 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-abd9497a-2f05-4c2e-b480-450d4216f7a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4b92fbe10f74e548fb54fafe00f2dac'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:15.795458 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-abd9497a-2f05-4c2e-b480-450d4216f7a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4b92fbe10f74e548fb54fafe00f2dac'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:15.805530 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-abd9497a-2f05-4c2e-b480-450d4216f7a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4b92fbe10f74e548fb54fafe00f2dac'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:15.806222 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-abd9497a-2f05-4c2e-b480-450d4216f7a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4b92fbe10f74e548fb54fafe00f2dac'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:15.818681 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-abd9497a-2f05-4c2e-b480-450d4216f7a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4b92fbe10f74e548fb54fafe00f2dac'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:15.825342 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1445/2887] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:59:15 2022] DELETE /identity/v3/users/b4b92fbe10f74e548fb54fafe00f2dac => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:15.830224 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42c906-ab5a-4f2d-a68c-1f8d7bb31555 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:15.830756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42c906-ab5a-4f2d-a68c-1f8d7bb31555 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:15.830756 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42c906-ab5a-4f2d-a68c-1f8d7bb31555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:15.851383 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-0c42c906-ab5a-4f2d-a68c-1f8d7bb31555 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:15.852680 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0c42c906-ab5a-4f2d-a68c-1f8d7bb31555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:15.935796 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1443/2888] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:15.948452 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9497a-2f05-4c2e-b480-450d4216f7a1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:15.950957 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-abd9497a-2f05-4c2e-b480-450d4216f7a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:16.023283 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9497a-2f05-4c2e-b480-450d4216f7a1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:16.027749 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cffb6-0d90-47fd-a89d-584cc11590a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:16.028393 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cffb6-0d90-47fd-a89d-584cc11590a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:16.028670 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cffb6-0d90-47fd-a89d-584cc11590a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:16.028932 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cffb6-0d90-47fd-a89d-584cc11590a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:16.039197 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cffb6-0d90-47fd-a89d-584cc11590a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:16.040682 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cffb6-0d90-47fd-a89d-584cc11590a3 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:16.051135 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1446/2889] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:16.430698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42c906-ab5a-4f2d-a68c-1f8d7bb31555 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:16.432338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-0c42c906-ab5a-4f2d-a68c-1f8d7bb31555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:16.444631 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42c906-ab5a-4f2d-a68c-1f8d7bb31555 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:16.452338 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549b8e7-2bc8-411d-bb3c-db229962d5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:16.453691 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549b8e7-2bc8-411d-bb3c-db229962d5b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:16.454045 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549b8e7-2bc8-411d-bb3c-db229962d5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:16.454498 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549b8e7-2bc8-411d-bb3c-db229962d5b3 admin admin] PATH_INFO: `/v3/projects/b3b2e41fde944cd3b16b9817e24a861c` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:16.455565 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549b8e7-2bc8-411d-bb3c-db229962d5b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3b2e41fde944cd3b16b9817e24a861c)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:16.457705 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549b8e7-2bc8-411d-bb3c-db229962d5b3 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:16.521611 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5549b8e7-2bc8-411d-bb3c-db229962d5b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3b2e41fde944cd3b16b9817e24a861c', 'request_id': 'req-5549b8e7-2bc8-411d-bb3c-db229962d5b3'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:16.527337 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5549b8e7-2bc8-411d-bb3c-db229962d5b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3b2e41fde944cd3b16b9817e24a861c', 'request_id': 'req-5549b8e7-2bc8-411d-bb3c-db229962d5b3'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:16.534086 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5549b8e7-2bc8-411d-bb3c-db229962d5b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3b2e41fde944cd3b16b9817e24a861c', 'request_id': 'req-5549b8e7-2bc8-411d-bb3c-db229962d5b3'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:16.549261 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5549b8e7-2bc8-411d-bb3c-db229962d5b3 admin admin] The token cache is being invalidate because project b3b2e41fde944cd3b16b9817e24a861c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:59:16.549546 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5549b8e7-2bc8-411d-bb3c-db229962d5b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:16.550229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-5549b8e7-2bc8-411d-bb3c-db229962d5b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:16.551783 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1444/2890] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:59:16 2022] DELETE /identity/v3/projects/b3b2e41fde944cd3b16b9817e24a861c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:16.556169 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cffb6-0d90-47fd-a89d-584cc11590a3 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:16.565998 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cffb6-0d90-47fd-a89d-584cc11590a3 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:16.627935 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a9ead-d743-419b-a7ca-d89591624114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:16.628626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a9ead-d743-419b-a7ca-d89591624114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:16.628896 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a9ead-d743-419b-a7ca-d89591624114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:16.629160 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a9ead-d743-419b-a7ca-d89591624114 admin admin] PATH_INFO: `/v3/users/6b1fa44d75854e1d8800bb39a36dd20d` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:16.630556 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a9ead-d743-419b-a7ca-d89591624114 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b1fa44d75854e1d8800bb39a36dd20d)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:16.632377 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a9ead-d743-419b-a7ca-d89591624114 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:16.692326 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-290a9ead-d743-419b-a7ca-d89591624114 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b1fa44d75854e1d8800bb39a36dd20d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:16.700610 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-290a9ead-d743-419b-a7ca-d89591624114 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b1fa44d75854e1d8800bb39a36dd20d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:16.704536 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:16.710209 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-290a9ead-d743-419b-a7ca-d89591624114 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b1fa44d75854e1d8800bb39a36dd20d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:16.710843 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-290a9ead-d743-419b-a7ca-d89591624114 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b1fa44d75854e1d8800bb39a36dd20d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:16.723473 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-290a9ead-d743-419b-a7ca-d89591624114 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b1fa44d75854e1d8800bb39a36dd20d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:16.729468 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1447/2891] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:59:16 2022] DELETE /identity/v3/users/6b1fa44d75854e1d8800bb39a36dd20d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:17.009881 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549b8e7-2bc8-411d-bb3c-db229962d5b3 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:17.011561 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5549b8e7-2bc8-411d-bb3c-db229962d5b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:17.025181 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549b8e7-2bc8-411d-bb3c-db229962d5b3 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:17.059388 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:17.094721 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:17.095573 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:17.095947 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:17.096221 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0 admin admin] PATH_INFO: `/v3/projects/a3b45b98bb5344d8a1355b52215c8abf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:17.097341 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3b45b98bb5344d8a1355b52215c8abf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:17.099380 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:17.155512 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3b45b98bb5344d8a1355b52215c8abf', 'request_id': 'req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.161069 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3b45b98bb5344d8a1355b52215c8abf', 'request_id': 'req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.168653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3b45b98bb5344d8a1355b52215c8abf', 'request_id': 'req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.183724 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0 admin admin] The token cache is being invalidate because project a3b45b98bb5344d8a1355b52215c8abf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:59:17.184092 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.184676 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.186195 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1445/2892] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:59:17 2022] DELETE /identity/v3/projects/a3b45b98bb5344d8a1355b52215c8abf => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:17.190229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a9ead-d743-419b-a7ca-d89591624114 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:17.191649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-290a9ead-d743-419b-a7ca-d89591624114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:17.204525 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a9ead-d743-419b-a7ca-d89591624114 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:17.267540 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5df11d-de61-4fb4-a35c-b2e613d409d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:17.268248 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5df11d-de61-4fb4-a35c-b2e613d409d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:17.268512 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5df11d-de61-4fb4-a35c-b2e613d409d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:17.268774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5df11d-de61-4fb4-a35c-b2e613d409d5 admin admin] PATH_INFO: `/v3/users/4e4a3c52ffcd4801b0eb05260f2d48d7` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:17.269981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5df11d-de61-4fb4-a35c-b2e613d409d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e4a3c52ffcd4801b0eb05260f2d48d7)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:17.271953 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5df11d-de61-4fb4-a35c-b2e613d409d5 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:17.324104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6f5df11d-de61-4fb4-a35c-b2e613d409d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e4a3c52ffcd4801b0eb05260f2d48d7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.330994 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6f5df11d-de61-4fb4-a35c-b2e613d409d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e4a3c52ffcd4801b0eb05260f2d48d7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.338831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6f5df11d-de61-4fb4-a35c-b2e613d409d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e4a3c52ffcd4801b0eb05260f2d48d7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.339447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6f5df11d-de61-4fb4-a35c-b2e613d409d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e4a3c52ffcd4801b0eb05260f2d48d7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.352269 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6f5df11d-de61-4fb4-a35c-b2e613d409d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e4a3c52ffcd4801b0eb05260f2d48d7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.358099 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1448/2893] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:59:17 2022] DELETE /identity/v3/users/4e4a3c52ffcd4801b0eb05260f2d48d7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:17.686254 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:17.687914 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:17.701816 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6c3a9-2d25-4832-9859-cb4cdfcdc4c0 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:17.773585 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db3094-395d-4380-9b26-858e5d63f2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:17.774259 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db3094-395d-4380-9b26-858e5d63f2c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:17.774543 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db3094-395d-4380-9b26-858e5d63f2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:17.774815 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db3094-395d-4380-9b26-858e5d63f2c9 admin admin] PATH_INFO: `/v3/projects/b83329f4ecd9462091615b0b2414b74f` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:17.775943 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db3094-395d-4380-9b26-858e5d63f2c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b83329f4ecd9462091615b0b2414b74f)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:17.777882 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db3094-395d-4380-9b26-858e5d63f2c9 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:17.832792 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-08db3094-395d-4380-9b26-858e5d63f2c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b83329f4ecd9462091615b0b2414b74f', 'request_id': 'req-08db3094-395d-4380-9b26-858e5d63f2c9'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.839721 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-08db3094-395d-4380-9b26-858e5d63f2c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b83329f4ecd9462091615b0b2414b74f', 'request_id': 'req-08db3094-395d-4380-9b26-858e5d63f2c9'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.840840 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:17.847525 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-08db3094-395d-4380-9b26-858e5d63f2c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b83329f4ecd9462091615b0b2414b74f', 'request_id': 'req-08db3094-395d-4380-9b26-858e5d63f2c9'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.866175 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-08db3094-395d-4380-9b26-858e5d63f2c9 admin admin] The token cache is being invalidate because project b83329f4ecd9462091615b0b2414b74f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:59:17.866468 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-08db3094-395d-4380-9b26-858e5d63f2c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.867143 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-08db3094-395d-4380-9b26-858e5d63f2c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.868771 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1446/2894] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:59:17 2022] DELETE /identity/v3/projects/b83329f4ecd9462091615b0b2414b74f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:17.873930 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db39ac2-e65c-4dcb-b974-d31255f3951c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:17.874435 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db39ac2-e65c-4dcb-b974-d31255f3951c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:17.874554 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db39ac2-e65c-4dcb-b974-d31255f3951c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:17.896019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7db39ac2-e65c-4dcb-b974-d31255f3951c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:17.897237 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7db39ac2-e65c-4dcb-b974-d31255f3951c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:17.951507 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:17.969981 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1449/2895] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:17.975557 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f38570-779f-4a73-ac42-0a4f80b8d808 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:17.975839 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f38570-779f-4a73-ac42-0a4f80b8d808 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:17.976160 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f38570-779f-4a73-ac42-0a4f80b8d808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:17.996027 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-07f38570-779f-4a73-ac42-0a4f80b8d808 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:17.997263 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-07f38570-779f-4a73-ac42-0a4f80b8d808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:18.005230 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1447/2896] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:18.009495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db39ac2-e65c-4dcb-b974-d31255f3951c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:18.019745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db39ac2-e65c-4dcb-b974-d31255f3951c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:18.025841 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b1a02-73e6-4ce8-ae51-8a6943822dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:18.026445 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b1a02-73e6-4ce8-ae51-8a6943822dfa admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:18.026718 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b1a02-73e6-4ce8-ae51-8a6943822dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:18.026979 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b1a02-73e6-4ce8-ae51-8a6943822dfa admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:18.027622 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b1a02-73e6-4ce8-ae51-8a6943822dfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:59:18.027892 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b1a02-73e6-4ce8-ae51-8a6943822dfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:18.029515 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b1a02-73e6-4ce8-ae51-8a6943822dfa admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:18.041382 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1450/2897] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:59:18.046050 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f38570-779f-4a73-ac42-0a4f80b8d808 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:18.056359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f38570-779f-4a73-ac42-0a4f80b8d808 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:18.061631 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9cbbd-3d6b-46fa-8ba7-62db19adbbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:18.062260 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9cbbd-3d6b-46fa-8ba7-62db19adbbe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:18.062529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9cbbd-3d6b-46fa-8ba7-62db19adbbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:18.062768 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9cbbd-3d6b-46fa-8ba7-62db19adbbe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:18.063416 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9cbbd-3d6b-46fa-8ba7-62db19adbbe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:18.065450 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9cbbd-3d6b-46fa-8ba7-62db19adbbe5 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:18.094133 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1448/2898] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:59:18 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:18.098482 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b1a02-73e6-4ce8-ae51-8a6943822dfa admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:18.110254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b1a02-73e6-4ce8-ae51-8a6943822dfa admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:18.177497 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5c677-f62c-4048-960f-47855df6d699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:18.178111 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5c677-f62c-4048-960f-47855df6d699 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:18.178361 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5c677-f62c-4048-960f-47855df6d699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:18.178624 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5c677-f62c-4048-960f-47855df6d699 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:18.179283 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5c677-f62c-4048-960f-47855df6d699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:18.181229 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5c677-f62c-4048-960f-47855df6d699 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:18.208210 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1451/2899] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:59:18 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:18.212830 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9cbbd-3d6b-46fa-8ba7-62db19adbbe5 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:18.224491 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9cbbd-3d6b-46fa-8ba7-62db19adbbe5 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:18.230313 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5713a5-45ce-4bd4-b1a4-b90cd36c7ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:18.230931 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5713a5-45ce-4bd4-b1a4-b90cd36c7ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:18.231203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5713a5-45ce-4bd4-b1a4-b90cd36c7ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:18.231463 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5713a5-45ce-4bd4-b1a4-b90cd36c7ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:18.232142 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5713a5-45ce-4bd4-b1a4-b90cd36c7ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:18.233772 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5713a5-45ce-4bd4-b1a4-b90cd36c7ca3 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:18.244727 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1449/2900] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:18.249239 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5c677-f62c-4048-960f-47855df6d699 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:18.261033 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5c677-f62c-4048-960f-47855df6d699 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:18.266802 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e6a49-e021-4b6e-91a8-eb45ffce2477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:18.267375 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e6a49-e021-4b6e-91a8-eb45ffce2477 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:18.267729 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e6a49-e021-4b6e-91a8-eb45ffce2477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:18.268023 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e6a49-e021-4b6e-91a8-eb45ffce2477 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:18.268774 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e6a49-e021-4b6e-91a8-eb45ffce2477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:18.270609 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e6a49-e021-4b6e-91a8-eb45ffce2477 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:18.282649 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1452/2901] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:18.287496 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5713a5-45ce-4bd4-b1a4-b90cd36c7ca3 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:18.302257 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5713a5-45ce-4bd4-b1a4-b90cd36c7ca3 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:18.307319 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4b69d-f922-440e-987c-4cae30c4faaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:18.307973 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4b69d-f922-440e-987c-4cae30c4faaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:18.308275 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4b69d-f922-440e-987c-4cae30c4faaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:18.308541 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4b69d-f922-440e-987c-4cae30c4faaa admin admin] PATH_INFO: `/v3/projects/1b8db666ebe644a7b42d74f5762f37ca/users/eeba043f2962456a99dfb81a99490f71/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:18.324698 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4b69d-f922-440e-987c-4cae30c4faaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b8db666ebe644a7b42d74f5762f37ca, user_id=eeba043f2962456a99dfb81a99490f71, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:18.327637 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4b69d-f922-440e-987c-4cae30c4faaa admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:18.343953 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1450/2902] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:18 2022] PUT /identity/v3/projects/1b8db666ebe644a7b42d74f5762f37ca/users/eeba043f2962456a99dfb81a99490f71/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:18.357104 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c63274-5be0-44bd-bdc7-58a5f78454c3 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:18.357472 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c63274-5be0-44bd-bdc7-58a5f78454c3 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:18.357724 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c63274-5be0-44bd-bdc7-58a5f78454c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:18.390140 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-79c63274-5be0-44bd-bdc7-58a5f78454c3 None None] MFA Rules not processed for user `eeba043f2962456a99dfb81a99490f71`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:18.391417 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-79c63274-5be0-44bd-bdc7-58a5f78454c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:18.473170 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1453/2903] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:18.478937 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4b69d-f922-440e-987c-4cae30c4faaa admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:18.552764 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4b69d-f922-440e-987c-4cae30c4faaa admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:18.558999 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013c046-c4ab-4001-b8ae-090f1fd32b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:18.559621 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013c046-c4ab-4001-b8ae-090f1fd32b9c admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:18.559899 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013c046-c4ab-4001-b8ae-090f1fd32b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:18.560203 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013c046-c4ab-4001-b8ae-090f1fd32b9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:18.560873 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013c046-c4ab-4001-b8ae-090f1fd32b9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:18.562891 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013c046-c4ab-4001-b8ae-090f1fd32b9c admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:18.593597 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1451/2904] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:59:18 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:18.599393 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c63274-5be0-44bd-bdc7-58a5f78454c3 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:18.611445 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c63274-5be0-44bd-bdc7-58a5f78454c3 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:18.681903 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:18.696900 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b5e91-23e1-4e7e-a6a9-b0f920da77c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:18.697635 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b5e91-23e1-4e7e-a6a9-b0f920da77c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:18.699343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b5e91-23e1-4e7e-a6a9-b0f920da77c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:18.699343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b5e91-23e1-4e7e-a6a9-b0f920da77c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:18.699343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b5e91-23e1-4e7e-a6a9-b0f920da77c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:18.701211 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b5e91-23e1-4e7e-a6a9-b0f920da77c4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:18.732655 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1454/2905] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:59:18 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:18.733220 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013c046-c4ab-4001-b8ae-090f1fd32b9c admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:18.750557 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013c046-c4ab-4001-b8ae-090f1fd32b9c admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:18.755594 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ddf71-48c6-4961-be3f-4bac80ec37fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:18.756312 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ddf71-48c6-4961-be3f-4bac80ec37fe admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:18.756706 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ddf71-48c6-4961-be3f-4bac80ec37fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:18.757077 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ddf71-48c6-4961-be3f-4bac80ec37fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:18.758014 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3ddf71-48c6-4961-be3f-4bac80ec37fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:18.759708 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3ddf71-48c6-4961-be3f-4bac80ec37fe admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:18.772741 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1452/2906] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:18.776983 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b5e91-23e1-4e7e-a6a9-b0f920da77c4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:18.793507 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b5e91-23e1-4e7e-a6a9-b0f920da77c4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:18.799783 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf734c0-ff9a-4159-b0fe-e692c248e636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:18.800823 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf734c0-ff9a-4159-b0fe-e692c248e636 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:18.800973 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf734c0-ff9a-4159-b0fe-e692c248e636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:18.801501 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf734c0-ff9a-4159-b0fe-e692c248e636 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:18.802785 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf734c0-ff9a-4159-b0fe-e692c248e636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:18.808074 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf734c0-ff9a-4159-b0fe-e692c248e636 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:18.830970 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1455/2907] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:18.838018 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ddf71-48c6-4961-be3f-4bac80ec37fe admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:18.849450 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ddf71-48c6-4961-be3f-4bac80ec37fe admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:18.854091 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf75048-aa40-44df-b583-43cf39a6d8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:18.854754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf75048-aa40-44df-b583-43cf39a6d8b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:18.855424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf75048-aa40-44df-b583-43cf39a6d8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:18.855424 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf75048-aa40-44df-b583-43cf39a6d8b3 admin admin] PATH_INFO: `/v3/projects/482d8718aa9740318e7626e2b3aea778/users/23090542abb64af9b2636c4e0aeea5a1/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:18.871875 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf75048-aa40-44df-b583-43cf39a6d8b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=482d8718aa9740318e7626e2b3aea778, user_id=23090542abb64af9b2636c4e0aeea5a1, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:18.874598 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf75048-aa40-44df-b583-43cf39a6d8b3 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:18.896900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1453/2908] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:18 2022] PUT /identity/v3/projects/482d8718aa9740318e7626e2b3aea778/users/23090542abb64af9b2636c4e0aeea5a1/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:18.912399 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1778f5e-c581-43e4-aa46-8f51db8885b0 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:18.913449 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1778f5e-c581-43e4-aa46-8f51db8885b0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:18.914096 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1778f5e-c581-43e4-aa46-8f51db8885b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:18.955174 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e1778f5e-c581-43e4-aa46-8f51db8885b0 None None] MFA Rules not processed for user `23090542abb64af9b2636c4e0aeea5a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:18.956538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e1778f5e-c581-43e4-aa46-8f51db8885b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:19.046576 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1456/2909] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:19.052556 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf75048-aa40-44df-b583-43cf39a6d8b3 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:19.096558 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:19.137565 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf75048-aa40-44df-b583-43cf39a6d8b3 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:19.143767 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926eaec-c92c-47a0-abab-67d9c4c5ece9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:19.144439 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926eaec-c92c-47a0-abab-67d9c4c5ece9 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:19.144716 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926eaec-c92c-47a0-abab-67d9c4c5ece9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:19.145111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926eaec-c92c-47a0-abab-67d9c4c5ece9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:19.145717 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d926eaec-c92c-47a0-abab-67d9c4c5ece9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:19.148389 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d926eaec-c92c-47a0-abab-67d9c4c5ece9 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:19.187754 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1454/2910] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:59:19 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:59:19.193379 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1778f5e-c581-43e4-aa46-8f51db8885b0 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:19.207845 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1778f5e-c581-43e4-aa46-8f51db8885b0 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:19.300961 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc055a-2e33-42cc-b835-b08a73323391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:19.301593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc055a-2e33-42cc-b835-b08a73323391 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:19.302662 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc055a-2e33-42cc-b835-b08a73323391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:19.302884 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc055a-2e33-42cc-b835-b08a73323391 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:19.303593 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc055a-2e33-42cc-b835-b08a73323391 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:19.306521 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc055a-2e33-42cc-b835-b08a73323391 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:19.332715 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1457/2911] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:59:19 2022] POST /identity/v3/users => generated 299 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:19.337107 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926eaec-c92c-47a0-abab-67d9c4c5ece9 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:19.348397 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926eaec-c92c-47a0-abab-67d9c4c5ece9 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:19.353529 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d05c7f-2e77-4d17-bc59-7850ed276365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:19.354135 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d05c7f-2e77-4d17-bc59-7850ed276365 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:19.354407 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d05c7f-2e77-4d17-bc59-7850ed276365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:19.354674 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d05c7f-2e77-4d17-bc59-7850ed276365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:19.355318 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d05c7f-2e77-4d17-bc59-7850ed276365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:19.356961 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d05c7f-2e77-4d17-bc59-7850ed276365 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:19.368435 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1455/2912] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:19.372859 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc055a-2e33-42cc-b835-b08a73323391 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:19.383716 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc055a-2e33-42cc-b835-b08a73323391 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:19.388545 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f0d8a-4f5f-40a4-8865-a5469bbec470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:19.389213 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f0d8a-4f5f-40a4-8865-a5469bbec470 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:19.389490 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f0d8a-4f5f-40a4-8865-a5469bbec470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:19.389759 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f0d8a-4f5f-40a4-8865-a5469bbec470 admin admin] PATH_INFO: `/v3/projects/76ad96a463694bd09db5b0c9a1220ef4/users/5cc4b12a8c154ab4bb3b136a9c1e025d/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:19.405869 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f0d8a-4f5f-40a4-8865-a5469bbec470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ad96a463694bd09db5b0c9a1220ef4, user_id=5cc4b12a8c154ab4bb3b136a9c1e025d, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:19.408713 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f0d8a-4f5f-40a4-8865-a5469bbec470 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:19.426958 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1458/2913] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:19 2022] PUT /identity/v3/projects/76ad96a463694bd09db5b0c9a1220ef4/users/5cc4b12a8c154ab4bb3b136a9c1e025d/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:19.431052 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d05c7f-2e77-4d17-bc59-7850ed276365 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:19.441912 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d05c7f-2e77-4d17-bc59-7850ed276365 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:19.505586 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc3339-f6bd-41e9-b284-7c8afef723ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:19.506272 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc3339-f6bd-41e9-b284-7c8afef723ec admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:19.506520 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc3339-f6bd-41e9-b284-7c8afef723ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:19.506785 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc3339-f6bd-41e9-b284-7c8afef723ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:19.507448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc3339-f6bd-41e9-b284-7c8afef723ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:19.509118 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc3339-f6bd-41e9-b284-7c8afef723ec admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:19.520153 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1456/2914] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:19.524714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f0d8a-4f5f-40a4-8865-a5469bbec470 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:19.536831 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f0d8a-4f5f-40a4-8865-a5469bbec470 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:19.542612 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2925b-c027-45f0-a698-53a339a03ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:19.543297 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2925b-c027-45f0-a698-53a339a03ff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:19.543574 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2925b-c027-45f0-a698-53a339a03ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:19.543842 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2925b-c027-45f0-a698-53a339a03ff5 admin admin] PATH_INFO: `/v3/projects/76ad96a463694bd09db5b0c9a1220ef4/users/5cc4b12a8c154ab4bb3b136a9c1e025d/roles/67ecca18fcea44ed8f6514715dcd8db8` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:19.545969 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2925b-c027-45f0-a698-53a339a03ff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ad96a463694bd09db5b0c9a1220ef4, user_id=5cc4b12a8c154ab4bb3b136a9c1e025d, role_id=67ecca18fcea44ed8f6514715dcd8db8)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:19.548995 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2925b-c027-45f0-a698-53a339a03ff5 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:19.562435 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1459/2915] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:19 2022] PUT /identity/v3/projects/76ad96a463694bd09db5b0c9a1220ef4/users/5cc4b12a8c154ab4bb3b136a9c1e025d/roles/67ecca18fcea44ed8f6514715dcd8db8 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:19.573438 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c338f5-c1d0-42d6-a403-c74c899a56fa None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:19.573719 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c338f5-c1d0-42d6-a403-c74c899a56fa None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:19.574136 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c338f5-c1d0-42d6-a403-c74c899a56fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:19.604448 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-26c338f5-c1d0-42d6-a403-c74c899a56fa None None] MFA Rules not processed for user `5cc4b12a8c154ab4bb3b136a9c1e025d`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:19.605683 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-26c338f5-c1d0-42d6-a403-c74c899a56fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:19.679231 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1457/2916] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:19.690963 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2925b-c027-45f0-a698-53a339a03ff5 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:19.693262 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2dc2925b-c027-45f0-a698-53a339a03ff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:19.708685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:19.756091 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2925b-c027-45f0-a698-53a339a03ff5 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:19.760240 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5650e-ddd8-46eb-8632-38fc7ea947b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:19.760938 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5650e-ddd8-46eb-8632-38fc7ea947b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:19.761393 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5650e-ddd8-46eb-8632-38fc7ea947b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:19.761658 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5650e-ddd8-46eb-8632-38fc7ea947b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:19.821309 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5650e-ddd8-46eb-8632-38fc7ea947b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:19.822773 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5650e-ddd8-46eb-8632-38fc7ea947b9 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:19.831980 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1460/2917] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:20.225007 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c338f5-c1d0-42d6-a403-c74c899a56fa None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:20.226461 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-26c338f5-c1d0-42d6-a403-c74c899a56fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:20.258644 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:20.287599 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c338f5-c1d0-42d6-a403-c74c899a56fa None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:20.291213 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a3d93-a711-41f5-9aba-856856ef7921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:20.291853 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a3d93-a711-41f5-9aba-856856ef7921 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:20.292157 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a3d93-a711-41f5-9aba-856856ef7921 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:20.292426 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a3d93-a711-41f5-9aba-856856ef7921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:20.294323 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f62a3d93-a711-41f5-9aba-856856ef7921 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:20.305759 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a3d93-a711-41f5-9aba-856856ef7921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:20.307198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a3d93-a711-41f5-9aba-856856ef7921 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:20.315676 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1458/2918] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:20.389726 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5650e-ddd8-46eb-8632-38fc7ea947b9 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:20.399630 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5650e-ddd8-46eb-8632-38fc7ea947b9 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:20.403252 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517aa3e-69ea-4d46-8fe6-046e55256e06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:20.403856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517aa3e-69ea-4d46-8fe6-046e55256e06 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:20.404173 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517aa3e-69ea-4d46-8fe6-046e55256e06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:20.404443 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517aa3e-69ea-4d46-8fe6-046e55256e06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:20.406251 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a517aa3e-69ea-4d46-8fe6-046e55256e06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:20.484438 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517aa3e-69ea-4d46-8fe6-046e55256e06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:20.485906 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517aa3e-69ea-4d46-8fe6-046e55256e06 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:20.497594 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1461/2919] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:21.394932 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a3d93-a711-41f5-9aba-856856ef7921 service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:21.397431 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f62a3d93-a711-41f5-9aba-856856ef7921 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:21.409305 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a3d93-a711-41f5-9aba-856856ef7921 service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:21.413855 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc197be-2a22-4f99-97ae-1f1d0d6b38b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:21.414489 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc197be-2a22-4f99-97ae-1f1d0d6b38b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:21.414712 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc197be-2a22-4f99-97ae-1f1d0d6b38b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:21.414995 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc197be-2a22-4f99-97ae-1f1d0d6b38b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:21.482415 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc197be-2a22-4f99-97ae-1f1d0d6b38b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:21.483929 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc197be-2a22-4f99-97ae-1f1d0d6b38b5 service cinder] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:21.496984 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1459/2920] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:22.091601 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517aa3e-69ea-4d46-8fe6-046e55256e06 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:22.101400 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517aa3e-69ea-4d46-8fe6-046e55256e06 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:22.105321 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b62883-82f9-4190-ba79-339678a4b2c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:22.105924 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b62883-82f9-4190-ba79-339678a4b2c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:22.106199 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b62883-82f9-4190-ba79-339678a4b2c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:22.106470 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b62883-82f9-4190-ba79-339678a4b2c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:22.115734 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b62883-82f9-4190-ba79-339678a4b2c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:22.117224 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b62883-82f9-4190-ba79-339678a4b2c3 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:22.127970 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1462/2921] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:26.067855 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc197be-2a22-4f99-97ae-1f1d0d6b38b5 service cinder] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:26.140269 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc197be-2a22-4f99-97ae-1f1d0d6b38b5 service cinder] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:26.145475 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bac043-425e-43c4-b011-febd1001b3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:26.146112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bac043-425e-43c4-b011-febd1001b3e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:26.146403 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bac043-425e-43c4-b011-febd1001b3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:26.146668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bac043-425e-43c4-b011-febd1001b3e0 admin admin] PATH_INFO: `/v3/users/eeba043f2962456a99dfb81a99490f71` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:26.148198 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bac043-425e-43c4-b011-febd1001b3e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeba043f2962456a99dfb81a99490f71)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:26.150169 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bac043-425e-43c4-b011-febd1001b3e0 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:26.200423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-09bac043-425e-43c4-b011-febd1001b3e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeba043f2962456a99dfb81a99490f71'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:26.201072 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-09bac043-425e-43c4-b011-febd1001b3e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeba043f2962456a99dfb81a99490f71'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:26.201665 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-09bac043-425e-43c4-b011-febd1001b3e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeba043f2962456a99dfb81a99490f71'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:26.210814 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-09bac043-425e-43c4-b011-febd1001b3e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeba043f2962456a99dfb81a99490f71'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:26.225157 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-09bac043-425e-43c4-b011-febd1001b3e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeba043f2962456a99dfb81a99490f71'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:26.226285 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:26.241206 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1460/2922] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:59:26 2022] DELETE /identity/v3/users/eeba043f2962456a99dfb81a99490f71 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:26.245959 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dce14-dadd-4ef6-990a-7e188bc9c26c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:26.246502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dce14-dadd-4ef6-990a-7e188bc9c26c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:26.246502 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dce14-dadd-4ef6-990a-7e188bc9c26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:26.266532 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-534dce14-dadd-4ef6-990a-7e188bc9c26c None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:26.267769 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-534dce14-dadd-4ef6-990a-7e188bc9c26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:26.304544 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:26.331620 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1463/2923] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:26.341688 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bac043-425e-43c4-b011-febd1001b3e0 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:26.343133 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-09bac043-425e-43c4-b011-febd1001b3e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:26.407359 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bac043-425e-43c4-b011-febd1001b3e0 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:26.413188 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0448259-617a-4d5a-8498-364ea7e170af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:26.413808 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0448259-617a-4d5a-8498-364ea7e170af service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:26.414086 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0448259-617a-4d5a-8498-364ea7e170af service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:26.414356 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0448259-617a-4d5a-8498-364ea7e170af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:26.425700 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0448259-617a-4d5a-8498-364ea7e170af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:26.427156 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0448259-617a-4d5a-8498-364ea7e170af service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:26.437799 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1461/2924] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:59:26.719572 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534dce14-dadd-4ef6-990a-7e188bc9c26c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:26.721302 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-534dce14-dadd-4ef6-990a-7e188bc9c26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:26.736299 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534dce14-dadd-4ef6-990a-7e188bc9c26c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:26.743761 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a8b40-1eed-487c-a995-75314d0abd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:26.744520 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a8b40-1eed-487c-a995-75314d0abd79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:26.744898 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a8b40-1eed-487c-a995-75314d0abd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:26.745058 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a8b40-1eed-487c-a995-75314d0abd79 admin admin] PATH_INFO: `/v3/projects/1b8db666ebe644a7b42d74f5762f37ca` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:26.746383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a8b40-1eed-487c-a995-75314d0abd79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b8db666ebe644a7b42d74f5762f37ca)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:26.748385 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a8b40-1eed-487c-a995-75314d0abd79 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:26.797279 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bc6a8b40-1eed-487c-a995-75314d0abd79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b8db666ebe644a7b42d74f5762f37ca', 'request_id': 'req-bc6a8b40-1eed-487c-a995-75314d0abd79'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:26.805751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bc6a8b40-1eed-487c-a995-75314d0abd79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b8db666ebe644a7b42d74f5762f37ca', 'request_id': 'req-bc6a8b40-1eed-487c-a995-75314d0abd79'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:26.811568 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bc6a8b40-1eed-487c-a995-75314d0abd79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b8db666ebe644a7b42d74f5762f37ca', 'request_id': 'req-bc6a8b40-1eed-487c-a995-75314d0abd79'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:26.822942 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bc6a8b40-1eed-487c-a995-75314d0abd79 admin admin] The token cache is being invalidate because project 1b8db666ebe644a7b42d74f5762f37ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:59:26.823227 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bc6a8b40-1eed-487c-a995-75314d0abd79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:26.823857 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bc6a8b40-1eed-487c-a995-75314d0abd79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:26.825383 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1464/2925] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:59:26 2022] DELETE /identity/v3/projects/1b8db666ebe644a7b42d74f5762f37ca => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:26.829483 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0448259-617a-4d5a-8498-364ea7e170af service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:26.839675 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0448259-617a-4d5a-8498-364ea7e170af service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:26.900254 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fc048-2a34-4921-956e-907a8817fb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:26.900900 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fc048-2a34-4921-956e-907a8817fb86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:26.901178 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fc048-2a34-4921-956e-907a8817fb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:26.901454 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fc048-2a34-4921-956e-907a8817fb86 admin admin] PATH_INFO: `/v3/users/23090542abb64af9b2636c4e0aeea5a1` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:26.902614 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980fc048-2a34-4921-956e-907a8817fb86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23090542abb64af9b2636c4e0aeea5a1)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:26.904657 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980fc048-2a34-4921-956e-907a8817fb86 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:26.955979 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-980fc048-2a34-4921-956e-907a8817fb86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23090542abb64af9b2636c4e0aeea5a1'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:26.956668 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-980fc048-2a34-4921-956e-907a8817fb86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23090542abb64af9b2636c4e0aeea5a1'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:26.957283 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-980fc048-2a34-4921-956e-907a8817fb86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23090542abb64af9b2636c4e0aeea5a1'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:26.965891 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-980fc048-2a34-4921-956e-907a8817fb86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23090542abb64af9b2636c4e0aeea5a1'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:26.978412 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-980fc048-2a34-4921-956e-907a8817fb86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23090542abb64af9b2636c4e0aeea5a1'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:26.991624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1462/2926] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:59:26 2022] DELETE /identity/v3/users/23090542abb64af9b2636c4e0aeea5a1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:27.277335 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a8b40-1eed-487c-a995-75314d0abd79 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:27.278659 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bc6a8b40-1eed-487c-a995-75314d0abd79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:27.291820 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a8b40-1eed-487c-a995-75314d0abd79 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:27.368664 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:27.369887 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:27.370221 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:27.370597 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e admin admin] PATH_INFO: `/v3/projects/482d8718aa9740318e7626e2b3aea778` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:27.372076 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=482d8718aa9740318e7626e2b3aea778)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:27.374158 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:27.428249 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '482d8718aa9740318e7626e2b3aea778', 'request_id': 'req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:27.438027 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '482d8718aa9740318e7626e2b3aea778', 'request_id': 'req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:27.445421 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '482d8718aa9740318e7626e2b3aea778', 'request_id': 'req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:27.446566 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:27.462340 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e admin admin] The token cache is being invalidate because project 482d8718aa9740318e7626e2b3aea778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:59:27.462632 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:27.463281 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:27.465096 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1465/2927] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:59:27 2022] DELETE /identity/v3/projects/482d8718aa9740318e7626e2b3aea778 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:27.468840 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fc048-2a34-4921-956e-907a8817fb86 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:27.470687 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-980fc048-2a34-4921-956e-907a8817fb86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:27.483706 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fc048-2a34-4921-956e-907a8817fb86 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:27.499650 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:27.553061 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9af54-1282-48f2-9d1b-df2390ae09b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:27.553936 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9af54-1282-48f2-9d1b-df2390ae09b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:27.554194 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9af54-1282-48f2-9d1b-df2390ae09b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:27.554565 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9af54-1282-48f2-9d1b-df2390ae09b6 admin admin] PATH_INFO: `/v3/users/5cc4b12a8c154ab4bb3b136a9c1e025d` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:27.555845 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9af54-1282-48f2-9d1b-df2390ae09b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cc4b12a8c154ab4bb3b136a9c1e025d)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:27.557868 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9af54-1282-48f2-9d1b-df2390ae09b6 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:27.608362 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a9b9af54-1282-48f2-9d1b-df2390ae09b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cc4b12a8c154ab4bb3b136a9c1e025d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:27.609029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a9b9af54-1282-48f2-9d1b-df2390ae09b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cc4b12a8c154ab4bb3b136a9c1e025d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:27.609803 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a9b9af54-1282-48f2-9d1b-df2390ae09b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cc4b12a8c154ab4bb3b136a9c1e025d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:27.617327 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a9b9af54-1282-48f2-9d1b-df2390ae09b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cc4b12a8c154ab4bb3b136a9c1e025d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:27.629908 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-a9b9af54-1282-48f2-9d1b-df2390ae09b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cc4b12a8c154ab4bb3b136a9c1e025d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:27.642867 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1463/2928] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:59:27 2022] DELETE /identity/v3/users/5cc4b12a8c154ab4bb3b136a9c1e025d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:27.937387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:27.939024 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:27.954695 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03dad43-66bf-4cd1-93fe-6dcb33825c0e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:28.017823 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc27e83-3614-42bd-ab69-7d9971c7dcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:28.018470 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc27e83-3614-42bd-ab69-7d9971c7dcd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:28.018745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc27e83-3614-42bd-ab69-7d9971c7dcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:28.019020 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc27e83-3614-42bd-ab69-7d9971c7dcd4 admin admin] PATH_INFO: `/v3/projects/76ad96a463694bd09db5b0c9a1220ef4` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:28.020315 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc27e83-3614-42bd-ab69-7d9971c7dcd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76ad96a463694bd09db5b0c9a1220ef4)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:28.023258 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc27e83-3614-42bd-ab69-7d9971c7dcd4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:28.072656 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bdc27e83-3614-42bd-ab69-7d9971c7dcd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76ad96a463694bd09db5b0c9a1220ef4', 'request_id': 'req-bdc27e83-3614-42bd-ab69-7d9971c7dcd4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:28.081902 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bdc27e83-3614-42bd-ab69-7d9971c7dcd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76ad96a463694bd09db5b0c9a1220ef4', 'request_id': 'req-bdc27e83-3614-42bd-ab69-7d9971c7dcd4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:28.089311 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bdc27e83-3614-42bd-ab69-7d9971c7dcd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76ad96a463694bd09db5b0c9a1220ef4', 'request_id': 'req-bdc27e83-3614-42bd-ab69-7d9971c7dcd4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:28.101538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bdc27e83-3614-42bd-ab69-7d9971c7dcd4 admin admin] The token cache is being invalidate because project 76ad96a463694bd09db5b0c9a1220ef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:59:28.101819 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bdc27e83-3614-42bd-ab69-7d9971c7dcd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:28.102464 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bdc27e83-3614-42bd-ab69-7d9971c7dcd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:28.103968 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1466/2929] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:59:27 2022] DELETE /identity/v3/projects/76ad96a463694bd09db5b0c9a1220ef4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:28.109251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9031e34-f6dd-45bd-b097-4e995fa23cf5 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:28.109534 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9031e34-f6dd-45bd-b097-4e995fa23cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:28.109806 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9031e34-f6dd-45bd-b097-4e995fa23cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:28.132835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-c9031e34-f6dd-45bd-b097-4e995fa23cf5 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:28.134090 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-c9031e34-f6dd-45bd-b097-4e995fa23cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:28.202391 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1464/2930] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:59:28.212554 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407789ed-de37-44ad-bac4-6708cbe074f7 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:28.212967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407789ed-de37-44ad-bac4-6708cbe074f7 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:28.213284 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407789ed-de37-44ad-bac4-6708cbe074f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:28.243856 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-407789ed-de37-44ad-bac4-6708cbe074f7 None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:28.245111 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-407789ed-de37-44ad-bac4-6708cbe074f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:28.254840 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1467/2931] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:28.260217 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9031e34-f6dd-45bd-b097-4e995fa23cf5 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:28.274416 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9031e34-f6dd-45bd-b097-4e995fa23cf5 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:28.279503 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5a228-cfa5-4da1-9053-c0df97d00ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:28.280142 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5a228-cfa5-4da1-9053-c0df97d00ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:28.280499 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5a228-cfa5-4da1-9053-c0df97d00ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:28.280768 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5a228-cfa5-4da1-9053-c0df97d00ac4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:28.281411 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5a228-cfa5-4da1-9053-c0df97d00ac4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:59:28.281720 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5a228-cfa5-4da1-9053-c0df97d00ac4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:28.283308 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5a228-cfa5-4da1-9053-c0df97d00ac4 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:28.295763 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1465/2932] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Aug 30 14:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:59:28.302167 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407789ed-de37-44ad-bac4-6708cbe074f7 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:28.311653 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407789ed-de37-44ad-bac4-6708cbe074f7 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:28.316387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177fca8-6ee8-45ba-87df-499bd3573c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:28.316986 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177fca8-6ee8-45ba-87df-499bd3573c1c admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:28.317275 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177fca8-6ee8-45ba-87df-499bd3573c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:28.317522 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177fca8-6ee8-45ba-87df-499bd3573c1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:28.318178 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177fca8-6ee8-45ba-87df-499bd3573c1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:28.320249 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177fca8-6ee8-45ba-87df-499bd3573c1c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:28.352730 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1468/2933] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Aug 30 14:59:28 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:59:28.357229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5a228-cfa5-4da1-9053-c0df97d00ac4 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:28.367903 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5a228-cfa5-4da1-9053-c0df97d00ac4 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:28.381474 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:28.442212 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61df2d-f711-4981-ac77-1722efceca65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:28.442830 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61df2d-f711-4981-ac77-1722efceca65 admin admin] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:28.443145 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61df2d-f711-4981-ac77-1722efceca65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:28.443370 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61df2d-f711-4981-ac77-1722efceca65 admin admin] PATH_INFO: `/v3/users` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:28.444325 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61df2d-f711-4981-ac77-1722efceca65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:28.446144 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61df2d-f711-4981-ac77-1722efceca65 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:28.470932 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1466/2934] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Aug 30 14:59:28 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:28.475629 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177fca8-6ee8-45ba-87df-499bd3573c1c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:28.490178 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177fca8-6ee8-45ba-87df-499bd3573c1c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:28.495272 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ea115-3e52-424c-83e4-85eca1a14596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:28.496026 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ea115-3e52-424c-83e4-85eca1a14596 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:28.496308 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ea115-3e52-424c-83e4-85eca1a14596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:28.496571 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ea115-3e52-424c-83e4-85eca1a14596 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:28.497180 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ea115-3e52-424c-83e4-85eca1a14596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:28.498802 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ea115-3e52-424c-83e4-85eca1a14596 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:28.511972 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1469/2935] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:28.515605 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61df2d-f711-4981-ac77-1722efceca65 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:28.526607 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61df2d-f711-4981-ac77-1722efceca65 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:28.532608 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac74a3f-8bc9-42f8-a267-086f4ed3b39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:28.533251 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac74a3f-8bc9-42f8-a267-086f4ed3b39e admin admin] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:28.533521 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac74a3f-8bc9-42f8-a267-086f4ed3b39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:28.533784 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac74a3f-8bc9-42f8-a267-086f4ed3b39e admin admin] PATH_INFO: `/v3/roles` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:28.534423 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac74a3f-8bc9-42f8-a267-086f4ed3b39e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:28.536111 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac74a3f-8bc9-42f8-a267-086f4ed3b39e admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:28.548295 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1467/2936] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:28.552926 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ea115-3e52-424c-83e4-85eca1a14596 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:28.563863 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ea115-3e52-424c-83e4-85eca1a14596 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:28.568861 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a38fe-5120-4ccb-9aff-02d82dc52f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:28.569764 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a38fe-5120-4ccb-9aff-02d82dc52f01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:28.570049 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a38fe-5120-4ccb-9aff-02d82dc52f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:28.570447 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a38fe-5120-4ccb-9aff-02d82dc52f01 admin admin] PATH_INFO: `/v3/projects/d1f949e4aa1f4db0b308b2999ef61f70/users/5e468e34d69e46439ca0b70f5723a689/roles/4eebd8098ab44bfc84cd3de29a679bdf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:28.590359 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a38fe-5120-4ccb-9aff-02d82dc52f01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f949e4aa1f4db0b308b2999ef61f70, user_id=5e468e34d69e46439ca0b70f5723a689, role_id=4eebd8098ab44bfc84cd3de29a679bdf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:28.593523 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a38fe-5120-4ccb-9aff-02d82dc52f01 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:28.610595 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1470/2937] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:28 2022] PUT /identity/v3/projects/d1f949e4aa1f4db0b308b2999ef61f70/users/5e468e34d69e46439ca0b70f5723a689/roles/4eebd8098ab44bfc84cd3de29a679bdf => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:28.622168 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b9a58-d97e-474f-bb85-11d6701c8bd4 None None] REQUEST_METHOD: `POST` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:28.622595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b9a58-d97e-474f-bb85-11d6701c8bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:28.623112 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b9a58-d97e-474f-bb85-11d6701c8bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:28.653374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.auth.core [None req-5c3b9a58-d97e-474f-bb85-11d6701c8bd4 None None] MFA Rules not processed for user `5e468e34d69e46439ca0b70f5723a689`. Rule list: `[]` (Enabled: `True`). {{(pid=81733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:28.654601 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-5c3b9a58-d97e-474f-bb85-11d6701c8bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:28.721191 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1468/2938] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:59:28.733514 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a38fe-5120-4ccb-9aff-02d82dc52f01 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:28.735413 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8f5a38fe-5120-4ccb-9aff-02d82dc52f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:28.759724 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:28.799525 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a38fe-5120-4ccb-9aff-02d82dc52f01 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:28.803834 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e6170-e03f-4154-82c1-f80fe192cbed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2788fece401a4a7ebdb516afe2dd8ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:28.804564 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e6170-e03f-4154-82c1-f80fe192cbed service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:28.804745 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e6170-e03f-4154-82c1-f80fe192cbed service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:28.804997 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e6170-e03f-4154-82c1-f80fe192cbed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:28.814082 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e6170-e03f-4154-82c1-f80fe192cbed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:28.815528 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e6170-e03f-4154-82c1-f80fe192cbed service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:28.825685 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1471/2939] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:31.450686 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b9a58-d97e-474f-bb85-11d6701c8bd4 None None] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:31.523944 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b9a58-d97e-474f-bb85-11d6701c8bd4 None None] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:31.528693 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8d872-fab2-45bd-9f0e-1131d343f6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:31.529339 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8d872-fab2-45bd-9f0e-1131d343f6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:31.529595 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8d872-fab2-45bd-9f0e-1131d343f6da admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:31.529878 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8d872-fab2-45bd-9f0e-1131d343f6da admin admin] PATH_INFO: `/v3/users/5e468e34d69e46439ca0b70f5723a689` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:31.531018 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8d872-fab2-45bd-9f0e-1131d343f6da admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e468e34d69e46439ca0b70f5723a689)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:31.532976 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8d872-fab2-45bd-9f0e-1131d343f6da admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:31.565413 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:31.586473 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-11c8d872-fab2-45bd-9f0e-1131d343f6da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e468e34d69e46439ca0b70f5723a689'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:31.587673 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-11c8d872-fab2-45bd-9f0e-1131d343f6da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e468e34d69e46439ca0b70f5723a689'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:31.588548 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-11c8d872-fab2-45bd-9f0e-1131d343f6da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e468e34d69e46439ca0b70f5723a689'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:31.597715 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-11c8d872-fab2-45bd-9f0e-1131d343f6da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e468e34d69e46439ca0b70f5723a689'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:31.614707 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-11c8d872-fab2-45bd-9f0e-1131d343f6da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e468e34d69e46439ca0b70f5723a689'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:31.628928 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1469/2940] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 14:59:31 2022] DELETE /identity/v3/users/5e468e34d69e46439ca0b70f5723a689 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:31.633826 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77267343-348f-46a5-b3a5-c4cfc1dd088d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:31.634671 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77267343-348f-46a5-b3a5-c4cfc1dd088d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:31.634851 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77267343-348f-46a5-b3a5-c4cfc1dd088d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:31.655495 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-77267343-348f-46a5-b3a5-c4cfc1dd088d None None] MFA Rules not processed for user `5f1827dde46f4915bfe3277a281b43bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:31.656805 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-77267343-348f-46a5-b3a5-c4cfc1dd088d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:31.728967 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1472/2941] 10.222.0.25 () {58 vars in 958 bytes} [Tue Aug 30 14:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:59:31.738902 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8d872-fab2-45bd-9f0e-1131d343f6da admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:31.740728 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-11c8d872-fab2-45bd-9f0e-1131d343f6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:31.810252 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8d872-fab2-45bd-9f0e-1131d343f6da admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:31.814827 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16ede2-ac58-47df-98bd-6d9d682a7ff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:31.815647 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16ede2-ac58-47df-98bd-6d9d682a7ff5 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:31.815931 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16ede2-ac58-47df-98bd-6d9d682a7ff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:31.816229 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16ede2-ac58-47df-98bd-6d9d682a7ff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:31.827309 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16ede2-ac58-47df-98bd-6d9d682a7ff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:31.828835 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16ede2-ac58-47df-98bd-6d9d682a7ff5 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:31.839445 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1470/2942] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:32.131233 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77267343-348f-46a5-b3a5-c4cfc1dd088d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:32.132762 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-77267343-348f-46a5-b3a5-c4cfc1dd088d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:32.151753 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77267343-348f-46a5-b3a5-c4cfc1dd088d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:32.157120 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:32.157936 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:32.158207 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:32.158497 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3 admin admin] PATH_INFO: `/v3/projects/d1f949e4aa1f4db0b308b2999ef61f70` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:32.159751 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f949e4aa1f4db0b308b2999ef61f70)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:32.161714 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:32.228029 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f949e4aa1f4db0b308b2999ef61f70', 'request_id': 'req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:32.237888 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f949e4aa1f4db0b308b2999ef61f70', 'request_id': 'req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:32.245292 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f949e4aa1f4db0b308b2999ef61f70', 'request_id': 'req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:32.256720 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3 admin admin] The token cache is being invalidate because project d1f949e4aa1f4db0b308b2999ef61f70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:59:32.257003 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:32.257689 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:32.259866 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1473/2943] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 14:59:32 2022] DELETE /identity/v3/projects/d1f949e4aa1f4db0b308b2999ef61f70 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:41.834374 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=45,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:42.253773 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:00:19.257476 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16ede2-ac58-47df-98bd-6d9d682a7ff5 service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:00:19.268046 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16ede2-ac58-47df-98bd-6d9d682a7ff5 service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:00:19.322080 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8f3b2-5212-460b-9cb7-0aed9e3feec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:00:19.322689 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8f3b2-5212-460b-9cb7-0aed9e3feec1 service neutron] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:00:19.322961 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8f3b2-5212-460b-9cb7-0aed9e3feec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:00:19.323233 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8f3b2-5212-460b-9cb7-0aed9e3feec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:00:19.325319 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-84d8f3b2-5212-460b-9cb7-0aed9e3feec1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:00:19.336934 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8f3b2-5212-460b-9cb7-0aed9e3feec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:00:19.337192 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8f3b2-5212-460b-9cb7-0aed9e3feec1 service neutron] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:00:19.347609 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1471/2944] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 15:00:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:00:29.342159 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=26 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:00:59.136387 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:00:59.138099 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:00:59.152646 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ab06c-6e65-4ffe-823a-494cb8922ed3 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:00:59.157019 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b0e8e-d20e-4cf1-a1bb-8466851c0036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:00:59.157625 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b0e8e-d20e-4cf1-a1bb-8466851c0036 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:00:59.157907 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b0e8e-d20e-4cf1-a1bb-8466851c0036 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:00:59.158175 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b0e8e-d20e-4cf1-a1bb-8466851c0036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:00:59.160633 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-476b0e8e-d20e-4cf1-a1bb-8466851c0036 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:00:59.227426 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b0e8e-d20e-4cf1-a1bb-8466851c0036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:00:59.229146 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b0e8e-d20e-4cf1-a1bb-8466851c0036 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:00:59.239002 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1474/2945] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 15:00:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:01:02.837029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8f3b2-5212-460b-9cb7-0aed9e3feec1 service neutron] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:01:02.838790 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-84d8f3b2-5212-460b-9cb7-0aed9e3feec1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:01:02.924183 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8f3b2-5212-460b-9cb7-0aed9e3feec1 service neutron] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:01:02.930070 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61218a4-4357-4510-970f-e33203b6a148 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95bbdafb0c4f34ad0662bc2e3ba4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:01:02.930679 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61218a4-4357-4510-970f-e33203b6a148 service nova] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:02.930954 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61218a4-4357-4510-970f-e33203b6a148 service nova] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:02.931246 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61218a4-4357-4510-970f-e33203b6a148 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:02.933008 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e61218a4-4357-4510-970f-e33203b6a148 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:01:02.945791 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61218a4-4357-4510-970f-e33203b6a148 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:01:02.947491 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61218a4-4357-4510-970f-e33203b6a148 service nova] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:01:02.958945 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1472/2946] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 15:01:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:01:04.304425 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b0e8e-d20e-4cf1-a1bb-8466851c0036 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:01:04.315684 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b0e8e-d20e-4cf1-a1bb-8466851c0036 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:01:04.320098 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a579d-78d4-44f8-9603-3fb47cf47a45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932fce741c5a4deba79a310a14e6bd2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:01:04.320692 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a579d-78d4-44f8-9603-3fb47cf47a45 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:04.320961 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a579d-78d4-44f8-9603-3fb47cf47a45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:04.321254 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a579d-78d4-44f8-9603-3fb47cf47a45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:04.323222 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f82a579d-78d4-44f8-9603-3fb47cf47a45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:01:04.398797 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a579d-78d4-44f8-9603-3fb47cf47a45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:01:04.400284 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a579d-78d4-44f8-9603-3fb47cf47a45 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:01:04.410910 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1475/2947] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 15:01:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:01:11.089851 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61218a4-4357-4510-970f-e33203b6a148 service nova] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:01:11.091515 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-e61218a4-4357-4510-970f-e33203b6a148 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:01:11.106994 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61218a4-4357-4510-970f-e33203b6a148 service nova] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:01:11.112832 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9addae-7660-49a8-b0e5-10555cc94de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:01:11.113451 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9addae-7660-49a8-b0e5-10555cc94de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:11.113727 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9addae-7660-49a8-b0e5-10555cc94de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:11.114009 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9addae-7660-49a8-b0e5-10555cc94de8 admin admin] PATH_INFO: `/v3/users/28b133b471d64de1b9088e6ab8a4c62d` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:11.115132 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9addae-7660-49a8-b0e5-10555cc94de8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28b133b471d64de1b9088e6ab8a4c62d)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:01:11.117140 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9addae-7660-49a8-b0e5-10555cc94de8 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:01:11.164483 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-ab9addae-7660-49a8-b0e5-10555cc94de8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28b133b471d64de1b9088e6ab8a4c62d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:11.165064 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-ab9addae-7660-49a8-b0e5-10555cc94de8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28b133b471d64de1b9088e6ab8a4c62d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:11.165659 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-ab9addae-7660-49a8-b0e5-10555cc94de8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28b133b471d64de1b9088e6ab8a4c62d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:11.174373 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-ab9addae-7660-49a8-b0e5-10555cc94de8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28b133b471d64de1b9088e6ab8a4c62d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:11.186629 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-ab9addae-7660-49a8-b0e5-10555cc94de8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28b133b471d64de1b9088e6ab8a4c62d'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:11.199625 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1473/2948] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 15:01:11 2022] DELETE /identity/v3/users/28b133b471d64de1b9088e6ab8a4c62d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:01:11.378513 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a579d-78d4-44f8-9603-3fb47cf47a45 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:01:11.389202 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a579d-78d4-44f8-9603-3fb47cf47a45 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:01:11.455014 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:01:11.455679 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:11.456206 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:11.456453 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c admin admin] PATH_INFO: `/v3/projects/f6ce53558466411292626efed88183b0` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:11.458198 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ce53558466411292626efed88183b0)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:01:11.459538 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:01:11.497408 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:01:11.507846 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ce53558466411292626efed88183b0', 'request_id': 'req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:11.516956 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ce53558466411292626efed88183b0', 'request_id': 'req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:11.523584 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ce53558466411292626efed88183b0', 'request_id': 'req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:11.539732 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c admin admin] The token cache is being invalidate because project f6ce53558466411292626efed88183b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:01:11.540126 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:11.540747 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:11.542343 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1476/2949] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 15:01:11 2022] DELETE /identity/v3/projects/f6ce53558466411292626efed88183b0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:01:11.547035 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9addae-7660-49a8-b0e5-10555cc94de8 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:01:11.548571 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-ab9addae-7660-49a8-b0e5-10555cc94de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:01:11.563682 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9addae-7660-49a8-b0e5-10555cc94de8 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:01:11.620274 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:01:11.627653 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e7b7e-ec85-475e-93c1-cf2e168597f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:01:11.628477 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e7b7e-ec85-475e-93c1-cf2e168597f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:11.628770 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e7b7e-ec85-475e-93c1-cf2e168597f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:11.629422 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e7b7e-ec85-475e-93c1-cf2e168597f9 admin admin] PATH_INFO: `/v3/users/119b7bf7dbae42c48426c01bcfb2688a` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:11.631063 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e7b7e-ec85-475e-93c1-cf2e168597f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=119b7bf7dbae42c48426c01bcfb2688a)` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:01:11.633051 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e7b7e-ec85-475e-93c1-cf2e168597f9 admin admin] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:01:11.685817 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4b2e7b7e-ec85-475e-93c1-cf2e168597f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '119b7bf7dbae42c48426c01bcfb2688a'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:11.686462 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4b2e7b7e-ec85-475e-93c1-cf2e168597f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '119b7bf7dbae42c48426c01bcfb2688a'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:11.687049 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4b2e7b7e-ec85-475e-93c1-cf2e168597f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '119b7bf7dbae42c48426c01bcfb2688a'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:11.695032 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4b2e7b7e-ec85-475e-93c1-cf2e168597f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '119b7bf7dbae42c48426c01bcfb2688a'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:11.708091 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.notifications [None req-4b2e7b7e-ec85-475e-93c1-cf2e168597f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '119b7bf7dbae42c48426c01bcfb2688a'} {{(pid=81733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:11.720519 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1474/2950] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Aug 30 15:01:11 2022] DELETE /identity/v3/users/119b7bf7dbae42c48426c01bcfb2688a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:01:11.906626 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:01:11.908277 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:01:11.922063 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf1e4e-b785-4e88-bacc-2e6180a1aa1c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:01:11.984294 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb56f49-107c-4f41-942f-5e1a5c7335be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1827dde46f4915bfe3277a281b43bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9ccb338f4e0425cad7e44cb87838ce1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:01:11.984852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb56f49-107c-4f41-942f-5e1a5c7335be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:11.985132 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb56f49-107c-4f41-942f-5e1a5c7335be admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:11.985403 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb56f49-107c-4f41-942f-5e1a5c7335be admin admin] PATH_INFO: `/v3/projects/9f56558480f446ffb7c52524676231f6` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:11.986535 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb56f49-107c-4f41-942f-5e1a5c7335be admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f56558480f446ffb7c52524676231f6)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:01:11.988517 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb56f49-107c-4f41-942f-5e1a5c7335be admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:01:12.165852 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-cdb56f49-107c-4f41-942f-5e1a5c7335be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f56558480f446ffb7c52524676231f6', 'request_id': 'req-cdb56f49-107c-4f41-942f-5e1a5c7335be'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:12.173748 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-cdb56f49-107c-4f41-942f-5e1a5c7335be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f56558480f446ffb7c52524676231f6', 'request_id': 'req-cdb56f49-107c-4f41-942f-5e1a5c7335be'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:12.180691 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-cdb56f49-107c-4f41-942f-5e1a5c7335be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f56558480f446ffb7c52524676231f6', 'request_id': 'req-cdb56f49-107c-4f41-942f-5e1a5c7335be'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:12.193578 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-cdb56f49-107c-4f41-942f-5e1a5c7335be admin admin] The token cache is being invalidate because project 9f56558480f446ffb7c52524676231f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:01:12.193868 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-cdb56f49-107c-4f41-942f-5e1a5c7335be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:12.194656 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-cdb56f49-107c-4f41-942f-5e1a5c7335be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:12.196481 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1477/2951] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Aug 30 15:01:11 2022] DELETE /identity/v3/projects/9f56558480f446ffb7c52524676231f6 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:01:21.717029 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=27,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:01:22.189708 dv-cinder-828805-5-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:DELETE=12,keystone:SELECT=62,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:03:23.631547 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e7b7e-ec85-475e-93c1-cf2e168597f9 admin admin] Authenticating user token {{(pid=81733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:23.633218 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-4b2e7b7e-ec85-475e-93c1-cf2e168597f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:23.806020 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e7b7e-ec85-475e-93c1-cf2e168597f9 admin admin] Validating token access rules against request {{(pid=81733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:23.883624 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10afaa2-9d7d-4463-8d8c-67cbd99fff89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cdd6b1ee9f4e97beb9bb037e446c8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9c57a581324217ad2e64a0c9c7fdee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:23.884537 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10afaa2-9d7d-4463-8d8c-67cbd99fff89 service placement] REQUEST_METHOD: `GET` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:23.884986 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10afaa2-9d7d-4463-8d8c-67cbd99fff89 service placement] SCRIPT_NAME: `/identity` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:23.885240 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10afaa2-9d7d-4463-8d8c-67cbd99fff89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:23.887113 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.fernet_utils [None req-f10afaa2-9d7d-4463-8d8c-67cbd99fff89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:23.900038 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10afaa2-9d7d-4463-8d8c-67cbd99fff89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:23.901645 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10afaa2-9d7d-4463-8d8c-67cbd99fff89 service placement] RBAC: Authorization granted {{(pid=81733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:23.913271 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: [pid: 81733|app: 0|req: 1475/2952] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Aug 30 15:03:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:03:33.908043 dv-cinder-828805-5-iscsi devstack@keystone.service[81733]: DEBUG dbcounter [-] [81733] Writing DB stats keystone:SELECT=89 {{(pid=81733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}